summaryrefslogtreecommitdiff
path: root/taskflow/patterns
Commit message (Collapse)AuthorAgeFilesLines
* Remove sixTakashi Kajinami2022-05-181-4/+3
| | | | | | | | This library no longer supports Python 2, thus usage of six can be removed. This also removes workaround about pickle library used in Python 2 only. Change-Id: I19d298cf0f402d65f0b142dea0bf35cf992332a9
* Update TaskFlow for networkx 2.xMichael Johnson2019-10-183-10/+11
| | | | | | | | | | | The networkx 2.x series has been out for two years now and supports python 3.6 and greater[1]. This patch updates TaskFlow to require a minimum of networkx 2.1. It also updates the code to support recent deprecation expiration introduced in the 2.4 release. [1] https://networkx.github.io/documentation/stable/news.html Change-Id: Ife31d353ba80824ebc63c8b21ee90943badc8da3
* Fix code to support networkx > 1.0Michal Arbet2018-07-111-1/+1
| | | | | | | | | | | | | With the release of NetworkX 2.0 the reporting API was moved to view/iterator model. Many methods were moved from reporting lists or dicts to iterating over the information. Methods that used to return containers now return views and methods that returned iterators have been removed in networkx. Because of this change in NetworkX 2.0 , taskflow code have to be changed also to support networkx > 2.0 Change-Id: I23c226f37bd85c1e38039fbcb302a2d0de49f333 Closes-Bug: #1778115
* Fix some misspellings in the function name and descriptionsPablo Iranzo Gómez2016-06-301-1/+1
| | | | Change-Id: I7e3451feb94b1f25b00c5e7b197bb6b527548306
* Allow for alterations in decider 'area of influence'Joshua Harlow2016-01-091-3/+25
| | | | | | | | Christmas came early. Closes-Bug: #1479466 Change-Id: I931d826690c925f022dbfffe9afb7bf41345b1d0
* Use graphs as the underlying structure of patternsJoshua Harlow2015-09-183-29/+41
| | | | | | | | | This unifies all the patterns to be graph based so that they are more uniform and there underlying constraints are more easy to understand (taskflow basically processes graphs). Change-Id: Ib2ab07c1c87165cf40a06508128010887f658391
* iter_nodes method added to flowsTimofey Durakov2015-08-283-3/+15
| | | | | | | New method allows to iterate over flow node and get access to node metadata during iteration. Change-Id: Ib0fc77f0597961602fbc3b49ba09e4df815d8230
* Improve docstrings in graph flow to denote exceptions raisedJoshua Harlow2015-08-051-4/+27
| | | | | | | | | | | Link the existing exceptions mentioned to there sphinx doc about them and also adds a section about when the 'DependencyFailure' exception is raised. Also tweaks the class docstring a little, to make it easier to understand. Change-Id: Ie4b989444c5ad73660cc61c0c3b2b1702b669087
* Add docs for u, v, decider on graph flow link methodJoshua Harlow2015-07-101-13/+28
| | | | | | | | | | | | | | The docstring is now quite important to know what the params are (they are no longer as obvious due to the decider addition) so add a useful docstring to that method. This also fixes how the targeted flow add was not passing allow **kwargs as it should have been and removes the duplicated docstrings that are in the child class (the parent class docstrings are just fine). Change-Id: Idacb7ee9f652fab4bdc762c1b49d7523e46e9a7b
* Add support for conditional executionJoshua Harlow2015-07-011-5/+13
| | | | | | | | | | | | | | | To make it possible to alter the runtime flow via a simple conditional like structure make it possible to have the graph flow link function take a decider that is expected to be some callable that will decide (via a boolean return) whether the edge should actually be traversed when running. When a decider returns false; the affected + successors will be set into the IGNORE state and they will be exempt from future runtime and scheduling decisions. Part of blueprint taskflow-conditional-execution Change-Id: Iab0ee46f86d6b8e747911174d54a7295b3fa404d
* Use 'node' terminology instead of 'item' terminologyJoshua Harlow2015-03-301-44/+47
| | | | | | | | | Since graphs are composed of 'nodes' it seems more appropriate to use that terminology (which is known to people with graph experience) instead of the non descriptive and/or standard 'item' terminology. Change-Id: I04a4521386d3cdf7e58fb9fa8cf26c00443c2cf6
* Make the graph '_unsatisfied_requires' be a staticmethodJoshua Harlow2015-03-251-6/+9
| | | | | | | | | | To make it possible for users to subclass this flow and replace this method (if they so choose to) make it a static method that is directed to the current module level function and call into the staticmethod when used (users can override staticmethods and change them in subclasses). Change-Id: I747c45636596ca5a8ad81dabcbba12ab55ce77d7
* Use ordered[set/dict] to retain orderingJoshua Harlow2015-03-111-2/+11
| | | | | | | | | | | | | | Instead of using always using a set/dict which do not retain use a ordered set and a ordered dict for requires, optional, and provides and rebind mappings types so that the ordering of these containers is maintained later when they are used. These ordering can be useful depending on the atom type (such as in a map and reduce tasks). Partial-Bug: 1357117 Change-Id: I365d11bbba4aa221bc36ca15441acecf199b4d56
* Use constants for link metadata keysJoshua Harlow2014-10-182-5/+5
| | | | | | | | | | | | Instead of using strings (which can be easy to mistype and get wrong), provide a set of constants that can be used to attach and use these keys in flows and at compilation. This also helps make it more clear what the keys do and where they are used. Change-Id: I5283b27617961136a4582bbcfce4617f05e8dd1d
* Relax the graph flow symbol constraintsJoshua Harlow2014-09-081-59/+97
| | | | | | | | | | In order to make it possible to have a symbol tree we need to relax the constraints that are being imposed by the graph flow. Part of blueprint taskflow-improved-scoping Change-Id: I2e14de2131c3ba4e3e4eb3108477583d0f02dae2
* Relax the unordered flow symbol constraintsJoshua Harlow2014-09-081-47/+2
| | | | | | | | | | | | In order to make it possible to have a symbol tree we need to relax and remove the constraints that are being imposed by the unordered constraints and later move those constraint checks and validations into the engines compilation stage. Part of blueprint taskflow-improved-scoping Change-Id: I80718b4bc01fbf0dce6a95cd2fac7e6e2e1814d1
* Relax the linear flow symbol constraintsJoshua Harlow2014-09-081-31/+2
| | | | | | | | | | | | In order to make it possible to have a symbol tree we need to relax and remove the constraints that are being imposed by the linear constraints and later move those constraint checks and validations into the engines compilation stage. Part of blueprint taskflow-improved-scoping Change-Id: I6efdc821ff991e83572d89f56be5c678d007f9f8
* Revamp the symbol lookup mechanismJoshua Harlow2014-09-081-1/+1
| | | | | | | | | | | | | | | | | | To complement the future changes in patterns we also want to allow the execution of patterns to be affected in a similar manner so that symbol lookup is no longer as confined as it was. This change adds in the following: - Symbol lookup by walking through an atoms contained scope/s. - Better error messaging when symbols are not found. - Adjusted & new tests (existing ones work). - Better logging of the symbol lookup mechanism (helpful during debugging, although it is very verbose...) Part of blueprint taskflow-improved-scoping Change-Id: Id921a4abd9bf2b7b5c5a762337f8e90e8f1fe194
* Be smarter about required flow symbolsJoshua Harlow2014-09-083-6/+46
| | | | | | | | | | | | | | Instead of blindly assuming all the symbols that are provided automatically work for all flows even if the flow has ordering constraints we should set the base flow class requires property to be abstract and provide flow specific properties that can do the appropriate analysis to determine what the flows unsatisfied symbol requirements actually are. Part of blueprint taskflow-improved-scoping Change-Id: Ie149c05b3305c5bfff9d9f2c05e7e064c3a6d0c7
* Upgrade hacking version and fix some of the issuesJoshua Harlow2014-06-131-3/+6
| | | | | | | | | | | | | | | | | Update hacking to the new requirements version and fix about half of the new reported issues. The other hacking issues are for now ignored until fixed by adjusting our tox.ini file. This commit fixes the following new hacking errors: H405 - multi line docstring summary not separated with an empty line E265 - block comment should start with '# ' F402 - import 'endpoint' from line 21 shadowed by loop variable Change-Id: I6bae61591fb988cc17fa79e21cb5f1508d22781c
* Put provides and requires code to basic FlowIvan A. Melnikov2014-05-073-62/+12
| | | | | | | | | | Code that calculates provides and requires for flow is almost identical for all patterns, so this change makes it completely identical and puts it to the base class. Other patterns are still allowed to override these properties for sake of customization or optimization. Change-Id: I6e875e863047b5287ec727fc9a491f252f144ecf
* Add a directed graph type (new types module)Joshua Harlow2014-04-201-10/+12
| | | | | | | | | | | | | | | | Most of the utility graph functions we have can be connected to a directed graph class that itself derives (and adds on to) the networkx base class. Doing this allows for functionality that isn't exposed in networkx to be exposed in our subclass (which is a useful pattern to have). It also makes it possible (if ever needed) to replace the networkx usage in taskflow with something else if this ever becomes a major request. Change-Id: I0a825d5637236d7b5dbdbda0d426adb0183d5ba3
* Fix spelling mistakeJoshua Harlow2014-03-261-1/+1
| | | | Change-Id: Ic3ae069565f0f9e25f243eead105e1b97fe437fc
* Adjust the exception hierachyJoshua Harlow2014-03-243-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Group the exceptions into the following groups * Storage * Jobs * Execution * Other (wrapped failure here) This grouping makes it easier to understand where one type of exception should be used vs using another type of exception. Backwards incompatible changes: * StorageError -> StorageFailure * AlreadyExists -> Duplicate * WrappedFailure now inherits from Exception and not directly from TaskFlowException since it wraps arbitrary other exceptions and is not specific to taskflow. Cleanups: * JobNotFound -> NotFound * EmptyFlow -> Empty * JobAlreadyExists -> AlreadyExists * InvariantViolation (X) * ConnectionFailure (X) Change-Id: I0e1e81b513fbbc7adb8bfaa1244993e345ab70d3
* Merge "Rework graph flow unit tests"Jenkins2014-03-241-11/+5
|\
| * Rework graph flow unit testsIvan A. Melnikov2014-03-221-11/+5
| | | | | | | | | | | | | | | | This commit adds unit tests that check graph flow methods without executing or flattening it. Now-redundant tests from other test suites are deleted. Change-Id: I8dafe0f9b295428831eddb3f9fd48f042d2f1ffc
* | Merge "Linear flow: mark links and rework unit tests"Jenkins2014-03-241-2/+1
|\ \ | |/
| * Linear flow: mark links and rework unit testsIvan A. Melnikov2014-03-211-2/+1
| | | | | | | | | | | | | | | | | | | | - add 'invariant': True to all links produced by linear flow; - write new unit tests for basic functionality of linear flow pattern; - remove some simple now-redundant unit tests from test_flow_dependencies. Change-Id: I1f4fa8a1dfc61485555a10e8d0629a67aab1997f
* | Merge "Drop indexing operator from linear flow"Jenkins2014-03-231-3/+0
|\ \
| * | Drop indexing operator from linear flowIvan A. Melnikov2014-03-211-3/+0
| |/ | | | | | | | | | | | | | | It is not needed. Also, using it complicates transitioning to another pattern (e.g. graph flow) and encourages bad programming practices. Change-Id: I9d0a4646a27df1a11ed436f26a0cb074b717647e
* | Merge "Removed copyright from empty files"Jenkins2014-03-231-15/+0
|\ \ | |/ |/|
| * Removed copyright from empty filesyangxurong2014-02-221-15/+0
| | | | | | | | | | | | | | | | | | According to policy change in HACKING: http://docs.openstack.org/developer/hacking/#openstack-licensing empty files should no longer contain copyright notices. Change-Id: I1f825add9036623ef626b48c66206ac2b452c592 Partial-Bug: #1262424
* | Iteration over links in flow interfaceIvan A. Melnikov2014-03-213-8/+29
| | | | | | | | | | | | | | | | | | In addition to iteration over its children (atoms or subflows) each pattern now provides iter_links() method that iterates over dependency links. This allows engines to treat all patterns in the same way instead knowing what structure each pattern expresses. Change-Id: I52cb5b0b501eefc8eb56a9ef5303aeb318013e11
* | Add retry to Flow patternsAnastasia Karpinska2014-03-143-8/+33
| | | | | | | | | | | | | | | | Implement dependency checks when adding items to flows: - retry can't be depended on tasks from it's subflow. - retry can't provide same values as tasks or other retries Change-Id: Iffa8e673fd2de39407ae22cd38ad523d484cbba7
* | Remove extra quote symbolIvan A. Melnikov2014-03-132-2/+2
|/ | | | | | Three quotes is just enough to open multiline string literal. Change-Id: I15d489f1a76be81a05b2378ef35120ee84d5eb04
* Remove extraneous vim configuration commentsyangxurong2014-02-144-8/+0
| | | | | | | | | Remove line containing comment - # vim: tabstop=4 shiftwidth=4 softtabstop=4 Change-Id: I7581cc88b8de433d5609ed06c6570b0b45c13573 Closes-Bug:#1229324
* Doc strings and comments clean-upskudriashev2014-01-263-3/+4
| | | | | | | | * Added missing period for doc strings * Correct syntax errors * Remove H402 from flake8 ignore list Change-Id: Ia8592bf99378e3658d6cca2ceb148bf9eb0b5de8
* Merge "Skip invariant checking and adding when nothing provided"Jenkins2014-01-242-1/+7
|\
| * Skip invariant checking and adding when nothing providedJoshua Harlow2013-12-312-1/+7
| | | | | | | | Change-Id: Icc3dca675820d331fb1d6e8f3a5a4169d1b2d113
* | Exceptions cleanupIvan A. Melnikov2014-01-142-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Most TaskFlow exception don't have 'Exception' suffix in their names, so we drop it from the few that have it, for sake of consistency and more concise and expressive code. MissingDependencies parent object is changed from InvalidState to InvariantViolation, which is more appropriate. Unused ClosedException is dropped. Breaking change: any client code that used renamed exceptions, ClosedException or relied on catching MissingDependencies as InvariantViolationException is affected. Change-Id: I649b7760d382ccb6df333bdffd667ba6b67a431e
* | Check tasks should not provide same valuesAnastasia Karpinska2014-01-022-1/+19
|/ | | | | | | | | | Check is added to linear_flow and unordered_flow patterns that there are no two child items provide same values. Unit tests added. Change-Id: I7a293d58a962de37ba2fd1dd989bf0a218417430 Closes-Bug: #1264995
* Don't forget to return selfJoshua Harlow2013-12-061-0/+2
| | | | | | | | To retain the functionality where we can chain function calls to add, link we need to return the object we are acting on (instead of none). Change-Id: Iadf0f9d3a21b6d079c1edd29063196180f5222ed
* Targeted graph flow patternIvan A. Melnikov2013-12-021-2/+63
| | | | | | | | | This change adds graph flow like pattern that is able to ignore graph nodes (not even add them to flow) if they are not needed to run certain task. Implements blueprint targeted-graph-flow Change-Id: I57d5a1c0598bd032d77e2a262b2d9644418ce3f6
* Remove uuid from taskflow.flow.FlowIvan A. Melnikov2013-10-153-6/+6
| | | | | | | In a way our resumption works it may not correspond uuid from flow details, and so it is hardly useful. Change-Id: I090d017e2f0f8475594af94a2430a34e6ed1ce70
* Pattern comment additions/adjustmentsJoshua Harlow2013-10-123-5/+8
| | | | Change-Id: I3154d1c228474d8699f3ae4d0be2fb46406a2f41
* Add reasons as to why the edges were createdJoshua Harlow2013-10-071-13/+28
| | | | | | | | | Reasons are useful for later analysis when running as to why the edge between two nodes was created so when linking items in the graph it would be nice to assign a reason. Change-Id: I2185cf5fb3c2c07c0f5536d3b210080c6f61d5dd
* Allow access by indexJoshua Harlow2013-10-041-0/+3
| | | | | | | | | Since a linear flow is a linear set of tasks or subflows we can easily add in an accessor that can fetch these items by index which is useful for indexing for any post-add operations. Change-Id: Ib1690584a08e24fae29285225e6c519b06ce2ddd
* Don't allow mutating operations on the underlying graphJoshua Harlow2013-09-301-39/+62
| | | | | | | | | | Instead of allowing a direct graph return of the underlying graph_flow graph we should return a frozen version instead so that users of the returned value can not mutate the graph without going through the graph_flow pattern (which could have undesired and harmful side-effects if this occurs). Change-Id: I38b35119d6e7bd7387b8ab467eba53aee5500629
* Task decorator was removed and examples updatedAnastasia Karpinska2013-09-232-4/+0
| | | | Change-Id: Ie49fe6c2f48a18130d1fd2a3aa5485cd8cee4ed4
* Remove the older (not used) resumption mechanismJoshua Harlow2013-09-222-154/+0
| | | | Change-Id: I0b6507a6f551c3f6d39e5310fb663beb51468ee0
View Lorry Controller Status