summaryrefslogtreecommitdiff
path: root/oslo_concurrency
Commit message (Collapse)AuthorAgeFilesLines
* Imported Translations from Zanata3.11.0OpenStack Proposal Bot2016-06-111-0/+101
| | | | | | | For more information about this automatic import see: https://wiki.openstack.org/wiki/Translations/Infrastructure Change-Id: I324acd475c84727b21aa9905c1184add135eb0cd
* Imported Translations from Zanata3.10.0OpenStack Proposal Bot2016-06-043-120/+19
| | | | | | | For more information about this automatic import see: https://wiki.openstack.org/wiki/Translations/Infrastructure Change-Id: Ifa683e2c40e33c05a4b6f6be514bb8a879179ee8
* Add reno for releasenotes managementKirill Bespalov2016-05-201-0/+18
| | | | | | | An initial patch to add reno and create a base directory for release notes Change-Id: Ie3f6653a27a49d8b2fced3f399b1d4c8f0c95b7c
* Merge "Add timeout option to ssh_execute"3.9.0Jenkins2016-05-142-3/+13
|\
| * Add timeout option to ssh_executezhangsong2016-05-122-3/+13
| | | | | | | | | | | | | | | | | | Add an optional timeout parameter to ssh_execute() method of oslo_concurrency.processutils. Timeout is None means infinity, which is the default. Change-Id: I45532799ecd23ff8c5572036d06725e71965b2b5 Closes-Bug: #1580826
* | Fix wrong import example in docstringChangBo Guo(gcb)2016-05-111-2/+2
|/ | | | Change-Id: I0abc5403851f9ab2f78cdf26d29b82c3ec76ec25
* Merge "processutils: add support for missing process limits"3.8.0Jenkins2016-05-043-11/+85
|\
| * processutils: add support for missing process limitsDaniel P. Berrange2016-04-193-11/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The original commit adding support for process limits only wired up address space, max files and resident set size limits. This is not sufficient to enable nova to protect qemu-img commands against malicious images. This commit adds support for the remaining limits supported by python: core file size, cpu time, data size, file size, locked memory size, max processes and stack size. Related-bug: #1449062 Change-Id: I164c4b35e1357a0f80ed7fe00a7ae8f49df92e31
* | Imported Translations from ZanataOpenStack Proposal Bot2016-04-218-50/+50
|/ | | | | | | For more information about this automatic import see: https://wiki.openstack.org/wiki/Translations/Infrastructure Change-Id: I65f5fc2140e9a12057343f89596f97648315e647
* Revert "Use tempfile.tempdir for lock_path if OSLO_LOCK_PATH is not set"Yuriy Taraday2016-02-261-4/+1
| | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 5021ef82fd8f0323b82d6d010bff9dab8a0cbcec. That commit set default location for lock files to a world-writable directory (like /tmp) which violates security precaution stated right in the help string of lock_path config variable: lock_path should be writable by the user of the process using it only. There is some history behind this: - when we decided to not do tempfile.mkdtemp if lock_path were not set (because it was broken), it broke upgrades for all components (starting with Cinder; - it was suggested to provide the same default value: https://review.openstack.org/60274 but it was not accepted because it weakens security; - there was a thread about this in ML, please read it for more history: http://lists.openstack.org/pipermail/openstack-dev/2013-December/021055.html it ended up with understanding that only a project using the library can provide sane default value for lock_path, not the library itself. Change-Id: I801004ad3c83862849696f00c6e7ca9877fd496b
* Use tempfile.tempdir for lock_path if OSLO_LOCK_PATH is not setSean M. Collins2016-02-251-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | DevStack spends a lot of time setting lock_path in a bunch of locations. http://codesearch.openstack.org/?q=oslo_concurrency%20lock_path&i=nope&files=&repos= The default for lock_path is an environment variable OSLO_LOCK_PATH. http://codesearch.openstack.org/?q=OSLO_LOCK_PATH&i=nope&files=&repos= Since the default is to consult an environment variable, which may not even be set, this means that really there is no default. If you do not set it - you end up getting errors like: http://paste.openstack.org/show/488108/ Ideally - a library should have a reasonable default, where if a user doesn't set something, the default will at least work. So, let's use Python's built in tempfile module, which has fairly complex rules for determining a sane directory to place temporary items in. https://docs.python.org/2/library/tempfile.html#tempfile.tempdir Change-Id: I6906af43bc0255cd215f2d9584ea000c81f5880e
* Make ProcessExecutionError picklableAngus Lees2016-02-192-4/+27
| | | | | | | | | | | Serialising/deserialising exceptions in python follows the protocol defined by pickle. It relies on the base Exception.__init__ setting self.args, and later resurrects exceptions with class(*args). This change rewrites processutils.ProcessExecutionError so it survives a pickle.dumps/loads round-trip. Change-Id: I9b8d104f60df868be7b808c72c932d08f1752777
* Merge "Add prlimit parameter to execute()"3.4.0Jenkins2016-02-013-0/+250
|\
| * Add prlimit parameter to execute()Victor Stinner2016-01-283-0/+250
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a new oslo_concurrency.prlimit module which is written to be used on the command line: python -m oslo_concurrency.prlimit --rss=RSS -- program arg1 ... This module calls setrlimit() to restrict the resources and then executes the program. Its command line is written to be the same than the Linux prlimit system program. Add a new ProcessLimits class processutils: resource limits on a process. Add an optional prlimit parameter to process_utils.execute(). If the parameter is used, wrap the command through the new oslo_concurrency prlimit wrapper. Linux provides a prlimit command line tool which implements the same feature (and even more), but it requires util-linux v2.21, and OpenStack targets other operating systems like Solaris and FreeBSD. Change-Id: Ib40aa62958ab9c157a2bd51d7ff3edb445556285 Related-Bug: 1449062
* | Update translation setupAndreas Jaeger2016-01-289-1/+502
|/ | | | | | | | | | | | | | | Follow new infra setup for translations, see spec http://specs.openstack.org/openstack-infra/infra-specs/specs/translation_setup.html for full details. This basically renames oslo.concurrency/locale/oslo.concurrency.pot to oslo_concurrency/locale/oslo_concurrency.pot. For this we need to update setup.cfg. Update also domain name in i18n.py. Change-Id: Ie774c4252f8481f2541ed0a2cad565d149dfcc72
* Add complementary remove lock with prefix functionJoshua Harlow2015-11-301-0/+28
| | | | | | | | | | | | It seems we provide (and projects use) the provided decorator 'synchronized_with_prefix' but we don't provide an equivalent function to complement that function to clean up its created lock files. Providing a complementary method seems pretty useful for projects that actually clean up these lock files (if any actually do in the first place). Change-Id: I601ce3992411e6a2ddded13aba4ac068cf8f14e2
* Drop python 2.6 support3.1.0ChangBo Guo(gcb)2015-11-231-2/+1
| | | | Change-Id: I921e29d25e0cfb089028f780bc942a60c9712278
* Use versionadded and versionchanged in docVictor Stinner2015-11-054-1/+48
| | | | | | | | | | | | | | | | Document in which version new types and functions were added using ".. versionadded:: x.y". Document changes using ".. versionchanged:: x.y." For watchdog module, add the versionadded tag in the module top docstring, not on each type/function. Add docstring to ssh_execute(). I used "git blame" + "git tag --contains=SHA1" to find these version, and then I checked manually each version. Change-Id: I56a7d8a4335833e9b49eedab026d20adfeedf942
* Fix Tests to run under OSXDavanum Srinivas2015-10-051-6/+9
| | | | Change-Id: I700b6de69411330b2a0fd8f2cdafa4026f85f1c7
* Merge "Use int enumerations for log error constants"Jenkins2015-09-141-14/+34
|\
| * Use int enumerations for log error constantsJoshua Harlow2015-09-011-14/+34
| | | | | | | | Change-Id: Iac6f4d860c220c653bbae17e7b1d69249259bd88
* | Use oslo_utils reflection to get 'f' callable nameJoshua Harlow2015-08-091-2/+5
|/ | | | | | | | | | | | | | | | | | | | | | | | Using the utility function gets a better name. For example: $ python >>> from oslo_utils import reflection >>> class A(object): ... def m(self): ... pass ... >>> z = A() >>> reflection.get_callable_name(z.m) '__main__.A.m' Versus: >>> z.m.__name__ 'm' Change-Id: I2daadd969383aaf49ad87876ba108dd80dd56f08
* Allow preexec_fn method for processutils.executeDavanum Srinivas2015-07-072-4/+33
| | | | | | | | | If the user specifies preexec_fn, we should call that in our existing _subprocess_setup. On windows, we silently drop this preexec_fn as subprocess.Popen raises a ValueError if we do pass it in. Change-Id: I0176c66fa2de001aa14f0d928d06fd894de55511
* processutils: ensure on_completion callback is always calledDaniel P. Berrange2015-07-022-9/+29
| | | | | | | | | | | If the subprocess.Popen.communicate method raises an exception, the on_completion callback is never invoked. If a caller is trying to use on_execute + on_completion to track lifecycle of a process this creates a problem, as they cannot reliably detect completion. Change-Id: I22b2d7bde8797276f7670bc289d915dab5122481 Closes-bug: #1470868
* Remove redundant fileutilsSteve Martinelli2015-06-303-149/+0
| | | | | | | fileutils no longer seems to be used, remove it instead of carrying it around. Change-Id: Idac62aa3dbf1d6897f1b7405b094cab0f9f79912
* Ensure we 'join' on the timer watchdog threadJoshua Harlow2015-06-221-0/+1
| | | | | | | | This object is just another thread and its cancel method does not join on itself, so we should make sure that we do to avoid leaving threads lying around. Change-Id: I37e3170b58627dee93513ec222f9a55001389dad
* Use better timing mechanisms instead of time.time()Joshua Harlow2015-06-223-12/+18
| | | | | | | | To ensure we show accurate timing information about lock acqusition and release times use a timing mechanism which can not/should not move backwards. Change-Id: I9559b20cf7de67fc474e6e17eda23791ecc4122e
* Merge "Fix LockFixture docstring"Jenkins2015-06-221-2/+2
|\
| * Fix LockFixture docstringIWAMOTO Toshihiro2015-06-161-2/+2
| | | | | | | | | | | | Fixtures.useFixture takes a class instance, not a class type. Change-Id: I48fff3f1a25b28e9ba15f79cbe18ea4c9582bd5c
* | Add 2 callbacks to processutils.execute()Tony Breeds2015-06-172-0/+32
|/ | | | | | | | | | | | | Add optional on_execute and on_completion callbacks to allow callers of procesutils.execute() to track process completion asynchronously. This could be used to cache the pid of long running tasks associated with an instance and then clear the cache when the process completes. While the tasks are running should it be required the pid retrieved and the process can be signaled. Co-Authored-By: abhishekkekane <abhishek.kekane@nttdata.com> Change-Id: Ifc23325eddb523f6449ba06a2deb0885a8a7009d
* Replace locks and replace with fasteners library provides onesJoshua Harlow2015-06-062-758/+4
| | | | | | | | | | | | | | | | | | | The fasteners library (extracted from this library and a couple other variations) provides the interprocess lock logic and the reader writer lock logic so we can remove the local version and we can just use it from that library instead. The tests that were ensuring the internals of this file lock have now moved to the repo where that library is (for the time being), currently travis is testing that repo against py2.6, py2.7 and py3.4. https://github.com/harlowja/fasteners/tree/master/fasteners/tests Docs also exist at: http://fasteners.readthedocs.org/en/latest/ Change-Id: I98565b22e68358efe28fea62f74f8ebfcc438ff7
* Sync from oslo-incubatorDavanum Srinivas2015-05-161-3/+6
| | | | | | | | | | Copy fileutils.py changes from change id(s): I369226f619895299c62f22a1debd0d4d120c912d I7decb7bf5f3185f7d26ad99b0cb3475a88d3ec99 Also fix the openstack-common.conf to the correct package name Change-Id: Ic77262bf94408395e720490264bcf7022461fe4d
* Add binary parameter to execute and ssh_execute1.9.0Victor Stinner2015-04-192-25/+105
| | | | | | | | | | | | | | | | Add an optional binary parameter to execute() and ssh_execute() functions of oslo_concurrency.processutils. If binary is True, stdout and stderr are returned as byte strings on Python 2 and Python 3. When Nova will be ported to Python 3, binary=True will be needed to get binary outputs (like an encryption key, see the bug #1410348). Add new tests. This changeset does not change the default behaviour. Change-Id: I283cadf04781942b65f211237070cc2354939cdb Related-Bug: 1410348
* Port processutils to Python 3Victor Stinner2015-04-182-18/+126
| | | | | | | | | | | | | | | | | | | | On Python 3, execute() and ssh_execute() now return stdout and stderr as Unicode. os.fsdecode() is used to stdout and stderr, it uses the locale encoding with the surrogateescape error handler. subprocess.Popen(universal_encoding=True) uses the same encoding and error handler. This change breaks the backward compatibility, but I believe that Unicode is more convinient in Python 3 and it is what users expect. In practice, there is no known (OpenStack) application compatible with Python 3 using oslo_concurrency.processutils yet. Add new tests. This changeset does not change the behaviour on Python 2, only Python 3 is impacted. Change-Id: Ibb510642c1440e2fbe7ad53d797bc7c32a3cbddf
* Merge "Switch to non-namespaced module imports"1.8.0Jenkins2015-03-121-1/+1
|\
| * Switch to non-namespaced module importsDavanum Srinivas2015-03-121-1/+1
| | | | | | | | Change-Id: Ib690450269c281409a095ed44bf5791d6c2a7cc5
* | Add lockutils.get_lock_path() functionDoug Hellmann2015-03-112-1/+32
|/ | | | | | | | | Provide an API to discover the path being used to store external lock files. Tempest will use this to set up some additional locking and to test its own behavior. Other projects might use it for error reporting or other purposes. Change-Id: Iad40c67072333cc25a6d3e39d7535ff14b573504
* Merge "processutils: execute(): fix option incompatibility"Jenkins2015-02-171-1/+6
|\
| * processutils: execute(): fix option incompatibilityCsaba Henk2015-02-141-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Issue: simultaneous usage of 'shell' and 'run_as_root' options led to error. Cause: it's actually a concealed TypeError: in 'shell' mode the command argument is assumed to be a string, elsewhere a list/tuple. Fix: the command editing implied by 'run_as_root' is performed with care taken of above tacit type assumption. Change-Id: Iaba33e6fda67793ae2874ba278c990271ee5e47f Closes-Bug: #1382873
* | Merge "Drop use of namespaced oslo.i18n"Jenkins2015-02-171-2/+2
|\ \
| * | Drop use of namespaced oslo.i18nIan Cordasco2015-01-301-2/+2
| | | | | | | | | | | | | | | | | | Related-blueprint: drop-namespace-packages Change-Id: Ic8247cb896ba6337932d7a74618debd698584fa0
* | | Ability to set working directory1.5.0Davanum Srinivas2015-02-132-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently folks end up using os.chdir() (see bug listed below) as we are not exposing the subprocess.Popen's cwd parameter. It's better to add a cwd parameter in processutils.execute so folks do not have to issue chdir() before and after the processutils. execute and there's less scope for problem as documented in the bug. Closes-Bug: #1414530 Change-Id: Ia4c77593c0f8301e059b349290e8663614a7ccfd
* | | Add eventlet test check to new tests __init__.pyBen Nemec2015-02-061-0/+19
|/ / | | | | | | | | | | | | | | | | | | | | This is used to allow us to test both with and without eventlet. It wasn't copied to the new tests directory in the namespace move, which means that once the old tests go away eventlet won't be monkey patching even when we try to make it. Also, there's a race depending on whether the top-level tests are run first, or the new package ones. Change-Id: Ie2705c3d21625e594471f27eefc3ac3147390cb7
* | Update Oslo imports to remove namespace packageDoug Hellmann2015-01-214-6/+6
| | | | | | | | | | Change-Id: I4ec9b2a310471e4e07867073e9577731ac34027d Blueprint: drop-namespace-packages
* | Merge "Revert "Port processutils to Python 3""1.4.1Jenkins2015-01-142-28/+9
|\ \
| * | Revert "Port processutils to Python 3"Doug Hellmann2015-01-132-28/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 7c7493feb53429577efca2c4b0380af03ddc149b. This change breaks the API of execute() and breaks nova's tests. Closes-Bug: #1410348 Change-Id: I33fa9b4e6099f521e0de3cb4585016fd694c4548
* | | Merge "Bump to hacking 0.10"0.4.0Jenkins2015-01-131-1/+1
|\ \ \
| * | | Bump to hacking 0.10Ben Nemec2015-01-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Only had to add a couple of newlines to the end of files. Change-Id: Ie7c31b3c9fc4ac00e8e3281888ab9faf84bc19c5
* | | | Merge "make time format for processutils match lockutils"Jenkins2015-01-111-1/+1
|\ \ \ \ | |_|_|/ |/| | |
| * | | make time format for processutils match lockutilsSean Dague2015-01-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lockutils uses %0.3fs for time format, which gives us a convenient time in milliseconds. Process utils was an unbounded string which gives us times like: CMD "cat /proc/31599/cmdline" returned: 0 in 0.00526595115662s execute The super long time strings are mostly confusing, and millisecond accuracy should be sufficient for processutils. Change-Id: I57f21b18ab060a36c3d88b0c5a94430eafeb02a2
View Lorry Controller Status