summaryrefslogtreecommitdiff
path: root/heat
Commit message (Collapse)AuthorAgeFilesLines
* Merge "Ignore spurious nested stack locks in convergence" into stable/ocataZuul2018-10-241-4/+11
|\
| * Ignore spurious nested stack locks in convergenceZane Bitter2018-09-071-4/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Several operations (e.g. stack check) are yet to be converted to convergence-style workflows, and still create locks. While we try to always remove dead locks, it's possible that if one of these gets left behind we won't notice, since convergence doesn't actually use stack locks for most regular operations. When doing _check_status_complete() on a nested stack resource, we wait for the operation to release the nested stack's lock before reporting the resource completed, to ensure that for legacy operations the nested stack is ready to perform another action on as soon as the resource is complete. For convergence stacks this is an unnecessary DB call, and it can lead to resources never completing if a stray lock happens to be left in the database. Only check the nested stack's stack lock for operations where we are not taking a resource lock. This corresponds exactly to legacy-style operations. This is modified from master to inline the _should_lock_on_action() method, which does not exist on this branch. Change-Id: I4eb20ad82cc3e9434da34500fafa3880567d0959 Story: #1727142 Task: 24939 (cherry picked from commit 83ae156927f879af59d7db518f888280ef369428)
* | Unit tests: Fix mock errors with too few side effectsZane Bitter2018-08-104-76/+76
|/ | | | | | | | | | | | | | | | | | | | | | | When a mock hasn't been provided with a long-enough list of side-effects for the number of times it is called, it raises StopIteration (instead of something sensible like AssertionError). Prior to Python 3.7, this exception could bubble up until it just stopped a task. (In Python 3.7 it will eventually be caught and turned into a RuntimeError.) To ensure that any errors of this sort are not handled silently, and to enable us to test for them prior to using Python 3.7, catch any StopIteration errors coming from plugin-provided non-generator functions and convert them to RuntimeError exceptions. This reveals many errors in the unit tests, which are also fixed by this patch. This backport contains fewer fixes than master, because many of the errors were introduced in the process of moving from mox to mock in the Rocky release cycle. Change-Id: I5a1eff6b704dff7c17edcbbe58cdbc380ae6abc9 Story: #2003412 Task: 24553 (cherry picked from commit 38fad07c0a01072ee4e17815f6ad0e5c2ce1f64a)
* Merge "Ignore NotFound error in prepare_for_replace" into stable/ocataocata-em8.0.7Zuul2018-08-082-5/+26
|\
| * Ignore NotFound error in prepare_for_replacerabi2018-07-312-5/+26
| | | | | | | | | | | | | | | | | | | | If the port is already deleted in neutron, we don't need to do anything. Change-Id: I9275610931a174919076330ec443f54f68cc1371 Story: #2003015 Task: 23035 (cherry picked from commit 3d542935fbe7def5e785c26a1b6a8b468cfeaa48)
* | Ignore RESOLVE translation errors when translating before_propsrabi2018-08-014-33/+68
|/ | | | | | | | | | | When translating name to id, if we don't find the resource it's better to ignore the error and return the non-translated value for before properties. Change-Id: Ic25c35098cff8b68e184a336b21f2e8df70bb1ee Story: #2003015 Task: 23103 (cherry picked from commit 8c39f3bb8b08ad37cb5411b448ef2acde6345d55)
* Generate user passwords with special charactersZane Bitter2018-04-066-20/+29
| | | | | | | | | | | | | When we create user accounts in order to allow signalling back to Heat, generate the passwords using a mix of upper- and lower-case letters, digits and special characters. This should pass most password_regex filters that users are likely to configure in Keystone, which was not true of the previous method (using only hex characters from a UUID). Change-Id: I7702d6ab550e4f1f53c4cea9f67ed0402afbd66d Closes-Bug: #1666129 Closes-Bug: #1444429 (cherry picked from commit 61412548aef0912b621861d08743821439172c6a)
* Fix entropy problems with OS::Random::StringZane Bitter2018-04-063-81/+248
| | | | | | | | | | | | | | | | | | | | | | | | | | | When generating a random string, once we had selected from the various required pools, we continued by selecting a pool at random and then selecting a character from that pool at random. This did not take into account the differing sizes of the available pools, nor the fact that the same character could appear in multiple pools, which resulted in a non-uniform probability distribution of characters. Since users mostly make use of this feature to generate default passwords for services they are deploying, this would result in the generated passwords having slightly less entropy than expected (and pathological cases were possible). Rectify this by always selecting non-constrained characters from a single combined pool, and by ensuring that each character appears only once in any pool we're selecting from. Since we also want to use this method to generate passwords for OpenStack Users, the new implementation is in a separate module in heat.common rather than mixed in with the resource's logic. Also, use a StringIO object to collect the characters rather than repeatedly appending to a string. Change-Id: Ia7b63e72c1e3c0649290caf4fea8a32f7f89560b Closes-Bug: #1757300 Related-Bug: #1666129 Related-Bug: #1444429 (cherry picked from commit 6e16c051ba9c2fc409c82fda19467d9ee1aaf484)
* Replace random with SystemRandom for RandomStringPavlo Shchelokovskyy2018-04-061-1/+5
| | | | | | | | | | | | | it might be theoretically possible to infer the state of standard Python's RNG in a long-running heat-engine service from multiple created RandomString resources. Let's use the random.SystemRandom (and os.urandom) for OS::Heat::RandomString instead. Change-Id: Iac5c03176fc8bae95ada883621196bd9cb453be3 Closes-Bug: #1745931 (cherry picked from commit 41605aaac1ec9fb0020c663b703255ee2cf3615f)
* Fix non-destructive upgrade for deprecated res types in convergenceZane Bitter2017-12-081-14/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a user updates from a deprecated resource type to an equivalent-but-differently-named one (e.g. from OS::Heat::SoftwareDeployments to OS::Heat::SoftwareDeploymentGroup), Heat is supposed to change the type without replacing the resource as it would normally do when a resource type changes. This was broken in convergence, because since 45073226752c58d640ea5a59b7e532c022a4939b the new Resource object we create during the check_resource operation (using the new type's plugin) is no longer automatically initialised with data from the database as resources in the legacy path are. Move the substitution checking to the Resource.load() method, so that it now returns an instance of the new plugin where allowed. In the actual update_convergence() method then we need only check that the resource class is the one we'd expect from the new template, and replace the resource if not. We do have a test that is designed to check that this is working, but in it we didn't compare the physical IDs of the resource that is potentially getting replaced, but rather the physical IDs of some other resource that can't possibly get modified (surprise! it doesn't change). Conflicts: heat/engine/resource.py This patch is modified with respect to the one on master, since the branch does not contain 960f626c2455b77e654aea1d79597fadb91dc894. Thus, resource_owning_stack.defn is substituted for initial_stk_defn and curr_stack.defn for latest_stk_defn. It is also modified with respect to the one on stable/pike, since the branch does not contain 764b8fb25132ac60c7760f134b78d600269ff516 or a7376f7494b310e9367ebe5dcb43b432a4053023. Thus, resource definitions are obtained from the Template rather than the StackDefinition. Change-Id: I75778abc303525a71d0a918f7192f00a43c21284 Closes-Bug: #1729439 (cherry picked from commit 93b4551d9a1ff9194b30a079f02db691129c69b0)
* Merge "Fix no message attribute in exception" into stable/ocata8.0.6Zuul2017-11-171-1/+3
|\
| * Fix no message attribute in exceptionricolin2017-07-271-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | For py35, message attribute in exception seems removed. We should directly get the string message from exception object if message attribute not presented. And since get message attribute already been deprecated. We should remove sopport on exception.message after we fully jump to py35. Partial-Bug: #1704725 Change-Id: I3970aa7c161aa82d179779f1a2f46405d5b0dddb (cherry picked from commit 4ba0b5fe7f31d4353e9c091b03df7324d1c20e88)
* | Merge "Remove the existing snapshots from the backend" into stable/ocataZuul2017-11-143-1/+45
|\ \
| * | Remove the existing snapshots from the backendhuangtianhua2017-11-133-1/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | We only have to remove the existing snapshots for resources when stack delete. Change-Id: Ia195f3c3380fe71e0888c8291209dd4562318951 Closes-Bug: #1716612 (cherry picked from commit 1861ad88670581e0658c8010950edd54ea4de422)
* | | Merge "Load all templates for generating parameter schema" into stable/ocataZuul2017-11-083-4/+41
|\ \ \
| * | | Load all templates for generating parameter schemaThomas Herve2017-08-173-4/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To be able to handle parameter defaults properly, we need to load all possible templates when merging the environments, to generate an approximate schema. Closes-Bug: #1669571 Change-Id: Idab781d3e6af92990c357a727aa60ec0aa8f5f05 (cherry picked from commit 7221cd2eba592cdb2097b3af52f9c297a54aae60)
* | | | Merge "Fix workflow update" into stable/ocataZuul2017-11-082-5/+14
|\ \ \ \ | |_|/ / |/| | |
| * | | Fix workflow updateThomas Herve2017-07-272-5/+14
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | Updating a workflow resource was broken because of some unexpected conditions filtering input, params and description. This removes those conditions and update the workflow regularly. Change-Id: I3bd187f8012bf4bddae1010fcf4a028c1829c644 (cherry picked from commit 721965f2ec7af7b584f13454cd5a41d68490dd47)
* | | Only validate properties once during create resource8.0.5ricolin2017-09-272-2/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While create a resource, we might retry to create or delete that resource. Within every action properties validate was act as pre-function before actually call any action. While the first creat run, if anything goes wrong during property validation, a validation error will raise right away. Which will definitely riase in the very first action run. But if it successed, it still required to revalidate each time an action triggered (whatever is creat or delete). This patch will ignore properties validate if it's not the first create run. Closes-Bug: #1691672 Change-Id: Ibf592a254a862613eddb77ea5933ec6ba0cd2d1a (cherry picked from commit 4a15c3387a258a66b366d6ace63c47192bb53fd1)
* | | Imported Translations from ZanataOpenStack Proposal Bot2017-09-2610-52/+71
| | | | | | | | | | | | | | | | | | | | | For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I5ee1f5311bf4fe8af0dd2ffe02ddf2209b8a74bb
* | | Merge "Cinder volume attachment attribute should not be cached" into ↵Jenkins2017-09-221-0/+1
|\ \ \ | | | | | | | | | | | | stable/ocata
| * | | Cinder volume attachment attribute should not be cachedGerry Buteau2017-08-171-0/+1
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Cinder volume 'attachments'attribute are dynamically built lists and must be queried from the Cinder API on each request. There are several factors that may change the contents of this list that are unknown to Heat. Caching should be prevented for this attribute. Closes-Bug: #1703605 Related-Bug: #1703387 Co-Authored-By: Paul Bourke <paul.bourke@oracle.com> Change-Id: I1a646e567b2d88f2acf09b3077c205c5b061d4bc (cherry picked from commit 9269787d39e052318ec59c58102cca128065b37a)
* | | Merge "Allow only validate external resource template" into stable/ocataJenkins2017-09-222-0/+22
|\ \ \
| * | | Allow only validate external resource templatericolin2017-08-182-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we choose only validate resource template, and external resource id was set, we should consider that external resource template is valid (since we only required `external_id` from that resource template format). So the result of this validate will be None (template validate passed). Closes-Bug: #1702977 Change-Id: I8b33c957a7185a0f52356f412b13609ba1a7b469 (cherry picked from commit 7cbf528da514e8830b9d1b211cf6d9554a9d6128)
* | | | Merge "Add validation of block_device_mapping_v2" into stable/ocataJenkins2017-09-152-79/+79
|\ \ \ \
| * | | | Add validation of block_device_mapping_v2huangtianhua2017-07-272-79/+79
| | |/ / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Heat assumes image/volume/snapshot in bdm_v2 as bootable sources, if user does not specify the 'boot_index' for them, heat will give the value '0' for all of them, but nova does not allow this. This patch adds validation to avoid the error. Closes-Bug: #1658007 Change-Id: I6809c8cb839eba20ffe12f783854fbaa7b9267d6 (cherry picked from commit 8fb7c3b9e43934c601bf78f2f8010aa3d6e8dc9e)
* | | | Merge "Add CACHE_NONE for FloatingIP attributes" into stable/ocataJenkins2017-09-141-2/+4
|\ \ \ \
| * | | | Add CACHE_NONE for FloatingIP attributesrabi2017-07-271-2/+4
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'port_id' and 'fixed_ip_address' attributes for FloatingIP would change when FloatingIPAssociation is created/updated. Therefore we should not cache/store them in db. Change-Id: I8cd580c15289a674888f194e8244041ed64bb172 Ralated-Bug: #1703605 (cherry picked from commit b0b2161ca98d6d177de12cc46a6be24c5b3b39d6)
* | | | Merge "Keep existing stack tags for patch update" into stable/ocataJenkins2017-09-143-2/+42
|\ \ \ \
| * | | | Keep existing stack tags for patch updaterabi2017-07-273-2/+42
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When updating a stack, we overwrite the existing tags. So if no tags are specified during update, all existing tags are removed. It would atleast be good to keep the existing tags when doing a patch update (with `--existing`). Change-Id: I910fe5237605405791eb6daa81c422bf7fd9eaa1 Closes-Bug: #1701098 (cherry picked from commit ed0beb6871b9a64c7a51ea85fd824e7c406343b3)
* | | | Merge "Don't set metadata for deleted resources" into stable/ocata8.0.4Jenkins2017-08-214-11/+42
|\ \ \ \
| * | | | Don't set metadata for deleted resourcesCrag Wolfe2017-04-254-11/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: I809dd844797dc0d9a8f31226a5b2da65a20a345d Partial-Bug: #1675286 (cherry picked from commit 8d7e3e41e8f02726dca33b5ec2f6d5b6b6b07a31)
* | | | | Merge "Fix snapshot delete in convergence" into stable/ocataJenkins2017-08-183-45/+9
|\ \ \ \ \
| * | | | | Fix snapshot delete in convergenceThomas Herve2017-07-053-45/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The patch at I1cb321a3878a0abce9b41832f76bf77c25bf7cb4 properly deleted the snapshots from the database, but as delete in convergence sets the template to an empty template, stack.resources is empty. This works around the problem by deleting the snapshots beforehand. Related-Bug: #1508299 Change-Id: Id1c2c1a293fdcda07c527f29fedc00b716b303bc (cherry picked from commit df1708b1a83f21f249aa08827fa88a25fc62c9e5)
* | | | | | Merge "Cache policy enforcer" into stable/ocataJenkins2017-08-182-1/+10
|\ \ \ \ \ \
| * | | | | | Cache policy enforcerThomas Herve2017-07-272-1/+10
| | |_|/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The policy enforcer can be used as a global object, there is no create one for every context creation. This caches the object in the module and reuse it in context. Change-Id: I03087b7bc139c8aaef95809043fc57f52b7900a1 Closes-Bug: #1658083 (cherry picked from commit 71c72340f58d28d7f865cb22917ecd18703cec74)
* | | | | | Merge "Log unhandled exceptions in worker" into stable/ocataJenkins2017-08-181-0/+16
|\ \ \ \ \ \ | |_|_|_|/ / |/| | | | |
| * | | | | Log unhandled exceptions in workerZane Bitter2017-07-271-0/+16
| |/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | RPC calls to the worker use 'cast', so nothing is listening to find out the result. If an exception occurs we will never hear about it. This change logs such unhandled exceptions as errors. Change-Id: I51365a9dee8fd4eff85e77d3e42bf33be814a22c Partial-Bug: #1703043 (cherry picked from commit 33a16aa7a808f3f1a9fc9faf2a8b1017a8bcbbbe)
* | | | | Merge "Mark stack failed when exception raised in resource check" into ↵Jenkins2017-08-171-11/+20
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | stable/ocata
| * | | | | Mark stack failed when exception raised in resource checkZane Bitter2017-08-151-11/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we get an unexpected exception when checking a resource, try to clean up by marking the stack as FAILED. The graph traversal will stop if we can't propagate any more RPC messages, so without this the stack would be stuck IN_PROGRESS indefinitely. Change-Id: I56ecfa7a9a328d1435c1f34ab14e56effb81bb21 Closes-Bug: #1703043 (cherry picked from commit ec1801931e13bb31e4fd0efd819b7cb8f4348c35)
* | | | | | Merge "Optimise the function.dep_attrs() function" into stable/ocataJenkins2017-08-171-1/+1
|\ \ \ \ \ \
| * | | | | | Optimise the function.dep_attrs() functionZane Bitter2017-07-261-1/+1
| | |/ / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Due to unfortunate copy-pasta, the original implementation of function.dep_attrs() iterates over items in a dictionary instead of values. Dictionary keys cannot have dependencies - they must be hashable and Function objects are not. So for each dictionary key we call dep_attrs() recursively 3 times as many times as we need to (for the key/value tuple, the key, and the value). For template snippets that are dominated by dictionaries, this means it took three times as long and used three times as much memory and twice as much stack space as it needed to. Change-Id: I13781540483daf88202d221a9f517746eebf0346 Partial-Bug: #1684272 (cherry picked from commit c889f08ab5279caae3d6fe4e0614fa938fb2c9ff)
* | | | | | Merge "Validate property values in nested stacks again" into stable/ocataJenkins2017-08-176-47/+46
|\ \ \ \ \ \
| * | | | | | Validate property values in nested stacks againZane Bitter2017-05-126-47/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In ced6f78aa065c1a7e6400c3be9ec3322e1e87416 we stopped doing validations of nested stacks at stack creation time, on the assumption that they had been validated when the parent stack was created. This assumption was incorrect; for children of the stack being created, the strict_validate global is always set during validation, so property values of resources will not be validated until it comes time to create the resource. Instead, prevent only redundant non-strict validations of stacks at nested depth 2 and greater. This means that every stack other than the root will be validated exactly twice - once without validating property values when the root is created, and again including property validation when the nested stack itself is created. Most of the performance benefits should remain; in the case of a large ResourceGroup using index substitution, we will now have to validate a lot of nearly-identical resource properties, however we still will not load into memory and validate a nested stack for each one as we originally did. Since that happens synchronously, it was likely the main contributor to RPC timeouts when dealing with large scaling groups. (During the validation at the creation of the root stack, only a single member of a ResourceGroup is validated even when index substitution is used. For scaling groups with identical members, only one member is validated since 3aebdabf2e78ac9e920b9dd8c748c4fad0d723c3.) This change reverts commit ced6f78aa065c1a7e6400c3be9ec3322e1e87416. Conflicts: heat/engine/stack.py Change-Id: I97cf789cee75931edef58b78c88f02da204d2a08 Closes-Bug: #1675589 Related-Bug: #1645336 (cherry picked from commit ea2673fb9a04588e0e294d159945d075657b9112)
* | | | | | | Merge "Fix OS::Zaqar::MistralSubscription with mistralclient 3.0" into ↵Jenkins2017-08-172-3/+7
|\ \ \ \ \ \ \ | |_|_|_|_|_|/ |/| | | | | | | | | | | | | stable/ocata
| * | | | | | Fix OS::Zaqar::MistralSubscription with mistralclient 3.0Zane Bitter2017-07-272-3/+7
| | |/ / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Starting with python-mistralclient 3.0.0, the Client object no longer has http_client as an attribute. Instead get it from the ExecutionManager. To ensure backward compatibility with earlier client versions, look for either the HTTPClient or the Client as an attribute of the ExecutionManager. Change-Id: I507284727039476acfdb1c9c8a26a360edc42417 Closes-Bug: #1704883 (cherry picked from commit 02176a581193b4ab60ad31e847ae3dd5f544c96a)
* | | | | | Merge "Handle post-delete hooks correctly" into stable/ocataJenkins2017-08-172-5/+6
|\ \ \ \ \ \
| * | | | | | Handle post-delete hooks correctlyThomas Herve2017-07-272-5/+6
| |/ / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This moves signal validation after hook handling, to be able to properly get signals during DELETE actions. Change-Id: I18e2284ab344f6c2e46e40744b771524b7bad9d9 Closes-Bug: #1690806 (cherry picked from commit 82fc15a540d598d29e23b815bd7bf3317a16b0ed)
* | | | | | Merge "Reorder the update task for nova server resource" into stable/ocataJenkins2017-08-171-8/+8
|\ \ \ \ \ \
| * | | | | | Reorder the update task for nova server resourcehuangtianhua2017-07-271-8/+8
| |/ / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Closes-Bug: #1689689 Change-Id: Ie3cba305bc6d16af21370d60025c5f3788b9b224 (cherry picked from commit 71b1c09afc5e26587d42723921eb2d45d91fe752)
View Lorry Controller Status