summaryrefslogtreecommitdiff
path: root/oslo_utils
Commit message (Collapse)AuthorAgeFilesLines
...
| * Fix regex to correctly recognize scientific notation with QemuImgInfoHervé Beraud2020-02-252-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | qemu 4.1.0 output shifts to scientific notation at 1000mb, breaking oslo.utils. Problem here is that the qemu-img output shifts to scientific notation: 999 => 999 MiB 1000 => 1e+03 MiB The regex in python-oslo-utils does not cover this. This issue is likely regexp parsing "disk size: 1e+03 MiB" value. These changes fix that. Change-Id: I4c016865890135023ceb497de18d75ccebd5961a Closes-Bug: 1864529
* | Flatten test caseStephen Finucane2020-02-251-86/+195
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | mock doesn't have a way to do the below pattern: foo.bar(call_arg).AndReturn(return_arg) my_call() Instead, you have to do the following: mock_foo.return_value = return_arg my_call() mock_foo.assert_call_once_with(call_arg) As such, we can't easily map the below to mock: def check_something(self): mock_foo.assert_call_once_with(call_arg) def test_foo(self): self.check_something() self.check_something() self.check_something() my_call() And we need to flatten this out, somewhat dumbly, to allow the conversion. Change-Id: I0f0e3e9ce4a410f60639c28aaadb9197aa47b492 Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* imageutils: Report format specific details when using JSON output formatLee Yarwood2020-02-142-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The data held within this section is particularly useful when dealing with encrypted LUKSv1, as shown below: $ qemu-img info --output=json rbd:volumes/volume { [..] "format-specific": { "type": "luks", "data": { "ivgen-alg": "plain64", "hash-alg": "sha256", "cipher-alg": "aes-256", "uuid": "de946b24-f318-4fa6-aacf-c5e97db04609", "cipher-mode": "xts", "slots": [ { "active": true, "iters": 1966316, "key-offset": 4096, "stripes": 4000 }, { "active": false, "key-offset": 262144 }, { "active": false, "key-offset": 520192 }, { "active": false, "key-offset": 778240 }, { "active": false, "key-offset": 1036288 }, { "active": false, "key-offset": 1294336 }, { "active": false, "key-offset": 1552384 }, { "active": false, "key-offset": 1810432 } ], "payload-offset": 2068480, "master-key-iters": 479636 } }, } Unfortunately the only documentation for this is in the QEMU codebase at present, for example for LUKSv1: https://github.com/qemu/qemu/blob/d8d5fefd8657d4f7b380b3a1533340434b5b9def/qapi/crypto.json#L272-L298 Related-Bug: #1861071 Change-Id: I133da07a5a9628b8a9338556939c858afae679f4
* trivial: Move setup code into setUp helperStephen Finucane2019-12-181-20/+7
| | | | | | | | All this stuff was being called at the beginning of each test. It belongs in 'setUp', so place it there. Change-Id: Icb50e64d47c364c3485371311abe1311502831a5 Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* Verify the sanitize keys are lowered3.42.1Dougal Matthews2019-11-222-3/+9
| | | | | | | This change is preventative to ensure any keys added in the future are all lowercase. Change-Id: Ib843fe59a80b081d9d0193717ff5a980e22c81b0
* Fix invalid escapes in regular expression stringsRadomir Dopieralski2019-11-211-3/+3
| | | | | | | | The "r" for raw strings has to repeated when several strings are concatenated, otherwise the backslash in the strings that miss the "r" are interpreted directly, and result in invalid escapes. Change-Id: I3736a40043c8aa6c36c21803bf68f842af5cd55e
* Make mask_dict_password case insensitive and add new patternsDougal Matthews2019-11-152-5/+22
| | | | | | | | | | | | | In Icc19b7c8bdb6a3182939d5e9fdef21288b19f43d mask_password was made case insensitive but mask_dict_password wasn't. This update makes the behaviour of these functions the same. Instead of lowering _SANITIZE_KEYS each time the source list is lowered. New password patterns from realworld logs were added to the patterns. Change-Id: Ic3ee301857630a15b9c26fd5d0fc907c43199517 Related-Bug: #1850843
* Make mask_password case insensitive, and add new patterns3.42.0Cédric Jeanneret2019-11-052-14/+56
| | | | | | | | | | | It appears that Mistral service logs everything, and doesn't use yet the mask_password (nor mask_dict_password) method. In order to ensure all is properly masked, we have to add some new patterns, and make it case insensitive in order to simplify and avoid duplicated entries. Change-Id: Icc19b7c8bdb6a3182939d5e9fdef21288b19f43d Related-Bug: #1850843 Signed-off-by: Cédric Jeanneret <cjeanner@redhat.com>
* Support "qemu-img info" virtual size in QEMU 4.1 and laterMark Mielke2019-09-202-2/+17
| | | | | | | | | | | | | | | | | | QEMU 4.0 and earlier have output like: virtual size: 1.5M (1572864 bytes) QEMU 4.1 and later have output like: virtual size: 1.5 MiB (1572864 bytes) Adjust the regular expression to allow for optional whitespace between the magnitude and the unit. Adjust the unit parsing to support the expanded "MiB" form. Change-Id: I1f316d6982c0def3296af4835484ad2d81a87fd4 Closes-Bug: 1844050
* Add digestmod when using hmac3.41.1Alfredo Moralejo2019-08-211-1/+3
| | | | | | | | | | | | | Until Python 3.8 hmc.new() defaulted the digestmod argument to 'hmac-md5'. This was deperecated, to be removed in Python 3.8 [1], so let's get ready for new python. Also switching to more secure sha1 algorithm, using md5 anywhere may trigger alerts from automatic security tools. [1] https://docs.python.org/3.8/library/hmac.html Change-Id: I4b365cb05de98bdd498b3c2094e4a77ab3944b12
* Merge "eventletutils: Optimise EventletEvent.clear()"Zuul2019-04-162-7/+27
|\
| * eventletutils: Optimise EventletEvent.clear()Zane Bitter2019-01-252-7/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When EventletEvent.clear() is called twice in succession without an intervening set(), there's no need to replace the underlying eventlet.event.Event object, since it has never been sent. Doing so would have woken other greenthreads waiting on the event to no particular end. When clear() is called after the event has been set(), we already did not do anything special with the existing eventlet.event.Event as we cannot call send() on it twice. We simply replace it with a new one; the code in wait() will handle the situation correctly, since it will wake up (due to the initial event having been sent) and begin waiting on the new eventlet.event.Event instead. This is consistent with the observed behaviour of threading.Event. A new unit test verifies this. Change-Id: Ibd5324926431fc760c3dd0be064324e3009cc2c2
* | Merge "Mask encryption_key_id"Zuul2019-03-272-1/+5
|\ \
| * | Mask encryption_key_idyenai2019-02-022-1/+5
| |/ | | | | | | | | Change-Id: I9e684cd8bab85728ff0117f95a30eb7dbb5bf51c Closes-Bug: #1814365
* | Update hacking versionZhijunWei2019-02-132-4/+4
|/ | | | | | | Use latest release 1.1.0 and compatible changes w.r.t pep8 Change-Id: Ide3a556918f341de6eafecb36ca431da40a0aed0 Closes-Bug: #1815715
* Avoid double-setting event3.40.2Ben Nemec2019-01-232-1/+13
| | | | | | | | | | | | | | | | In commit cc8b51e1e16f6bdc7d6c0e571e2002e70cde098d we added a send on the old event when clearing an EventletEvent. However, this was done unconditionally, which means if the event was already sent then we attempt to send it again. This fails with: AssertionError: Trying to re-send() an already-triggered event. Similar to 14a53c4d8a9d5605b14a503803859df0d6d4b820, we should check if self._set is True and if so then we know that the event was already sent and we don't need to do it again. Change-Id: I660601383072d11e4a077aada8c1b8c30b9d8d1d Closes-Bug: 1812922
* Merge "Avoid calling eventlet.event.Event.reset()"3.40.1Zuul2019-01-182-9/+6
|\
| * Avoid calling eventlet.event.Event.reset()Zane Bitter2019-01-032-9/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The eventlet maintainers have made it clear that calling reset() on an Event is not recommended: https://github.com/eventlet/eventlet/commit/715b2ced5239bbada84c06f488b9c8dda8180230 We don't really need to resend the event to wake up threads, because Event.wait() will return immediately once the event has been sent. So just rely on the internal _set flag to tell us whether we need to send the event. Change-Id: I466aa7cb64308e018598c3bb63a9d0cfbc833adc
* | Merge "Support non-dict mappings in mask_dict_password"3.40.0Zuul2019-01-082-3/+45
|\ \ | |/ |/|
| * Support non-dict mappings in mask_dict_passwordBen Nemec2018-12-032-3/+45
| | | | | | | | | | | | | | | | | | | | mask_dict_password doesn't actually have a dependency on the dict type specifically. It can work on any subclass of collections.Mapping. This changes the isinstance check to reflect that and adds a unit test using a collections.Mapping subclass. Change-Id: I28781acf027b9b34f8274196db5dd4d2a9adc9ba Closes-Bug: 1804528
* | Fix race condition in eventletutils EventZane Bitter2018-12-052-3/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The threading-compatible eventlet Event class has a race condition on the wait method. If greenthread A is blocked on the wait, but another greenthread B calls clear() and then set(), B calls self._event.send(), but A is waiting on a different eventlet Event which is no longer used by the oslo.service Event... To resolve this, when clearing an Event trigger the underlying eventlet Event immediately, then have the wait() method resume waiting on the new eventlet Event. Change-Id: I81579e2977bb965a5398a2cb4e3e24f5671e856a Co-Authored-By: Victor Stinner <vstinner@redhat.com> Co-Authored-By: Hervé Beraud <hberaud@redhat.com> Closes-Bug: #1805706
* | Don't use monotonic on Python >=3.3Zane Bitter2018-12-051-1/+5
| | | | | | | | | | | | | | | | | | | | | | A change to the global-requirements[1] has limited use of the monotonic library to Python versions earlier than 3.3 (later versions have built-in support for a monotonic clock), so no requirements changes can be merged until we similarly limit it in requirements.txt. [1] https://review.openstack.org/615441 Change-Id: Ib53d89ea820d21114d10280c9f4ab1b0c2c4a4bc
* | Expose eventlet Event wrapper class3.38.0Ben Nemec2018-11-012-3/+4
|/ | | | | | | | | | | | | | | We have a need to use the eventlet Event class directly in oslo.service. Currently it is copy-pasted from this project, but we now have a duplicate bug due to that so let's just expose it so one copy can be used in both places. This should be safe as the class implements a stdlib interface so we can't really alter the API, and it's not really private since instances of it are returned to users. Change-Id: If8e7a41f9fe5573a780f9faabdbaf1326631d37a Related-Change: https://review.openstack.org/558879 Related-Bug: 1800879
* Fix exception raise at rpdb session3.37.1BubaVV2018-09-071-2/+2
| | | | | | | | Some components raise unrelated exceptions during rpdb session due to unaware attribute handling of file-like objects Change-Id: I04dd5a7d0f6ec2920671dfa612439427dffefd52 Closes-bug: #1791104
* Merge "Fix docstring formatting nit in uuidsentinel"3.37.0Zuul2018-08-281-1/+1
|\
| * Fix docstring formatting nit in uuidsentinelBen Nemec2018-08-281-1/+1
| | | | | | | | | | | | | | | | This doesn't actually appear in the docs because it's a private class, but if we ever made it public we want it to format the code block correctly. Change-Id: Ia79b58d6d7847b65f946763c82a25ce8d8e581fb
* | Merge "UUID sentinel"Zuul2018-08-282-0/+62
|\ \ | |/
| * UUID sentinelEric Fried2018-08-282-0/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a private _UUIDSentinels() class to oslo_utils.fixture that behaves like a mock.sentinel, but returns specifically UUIDs. Expose a (singleton) instance of it called uuidsentinel. This is mostly copied from [1], with the following differences: - We don't do the enforced-singleton thing [2]. But importing the uuidsentinel global yields the same behavior. - We don't do the local import thing [3][4], because we're already in the right lib, so no issues with circular imports. - Locking is added to make this threadsafe. (See discussion at [5]) Note that there is some question as to whether it is more appropriate for this to live here or in oslotest [6]. This has been discussed on the dev ML [7] and it was concluded [8] that it should live here. [1] https://github.com/openstack/nova/blob/722d5b477219f0a2435a9f4ad4d54c61b83219f1/nova/tests/uuidsentinel.py [2] https://github.com/openstack/nova/blob/722d5b477219f0a2435a9f4ad4d54c61b83219f1/nova/tests/uuidsentinel.py#L30 [3] https://github.com/openstack/nova/blob/722d5b477219f0a2435a9f4ad4d54c61b83219f1/nova/tests/uuidsentinel.py#L18-L19 [4] https://github.com/openstack/nova/blob/722d5b477219f0a2435a9f4ad4d54c61b83219f1/nova/tests/uuidsentinel.py#L26 [5] http://eavesdrop.openstack.org/irclogs/%23openstack-oslo/%23openstack-oslo.2018-08-20.log.html#t2018-08-20T20:10:33 [6] https://review.openstack.org/#/c/594068/ [7] http://lists.openstack.org/pipermail/openstack-dev/2018-August/133670.html [8] http://lists.openstack.org/pipermail/openstack-dev/2018-August/133861.html Change-Id: I214ff21b461fa1ca4b83476e1d0a763efe986217
* | Remove moxstubout usageChuck Short2018-08-221-4/+5
|/ | | | | | | | As of oslotest version 3.5.0 moxstub will be deprecated, so remove it where it has been used. Change-Id: I9f038428cbea07a307f08c2251eafed7ab8bbea1 Signed-off-by: Chuck Short <chucks@redhat.com>
* Remove extra copy.deepcopyJoshua Harlow2018-07-191-4/+1
| | | | | | | | | This method is called recursively (ie deeply) by default so there doesn't seem to be a good reason to deepcopy over and over and over at every recusion level especially since a new output dictionary is getting created anyway. Change-Id: I644ef881e487c06dc4db77d60cfe765b0e59b547
* Merge "Handle non-string keys appropriately"3.36.4Zuul2018-07-182-5/+18
|\
| * Handle non-string keys appropriatelyJoshua Harlow2018-07-172-5/+18
| | | | | | | | | | | | | | | | | | | | | | | | In python dict keys can be anything that is hashable; which includes non-strings such as ints. Currently the code is blowing up with these types of keys with exceptions like: TypeError: argument of type 'int' is not iterable So to fix that handle the case where non-string keys are found. Change-Id: I4f576a089df6f68e43572bf0eee15e99f2b557fe
* | Fix exception with secretutilschangxun2018-05-242-22/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. There are some problems about the test method. problem 1: Unit tests may not cover our function, it depends on the python version that performed the test. problem 2: when using function 'constant_time_compare(first, second)', 'first' and 'second' params are usually HMAC digest values, it is not appropriate to use utf-8 encoded values as mock data. 2. The previous commit `f1d332a` lead into a bug, but due to the problem 1 and the problem 2, we did not find out the error. Change-Id: I1c29bfe69f8eda60f3c5caaf3e5447dd5b69b108 Closes-Bug: #1772851
* | Add private_key to the list of sanitized keys3.36.2Matthew Booth2018-05-111-1/+1
| | | | | | | | | | | | | | | | Nova's create keypair operation will currently log the generated keypair if debug logs are enabled. Closes-Bug: #1770683 Change-Id: I7f889f0bf254fad43b1e26d32fa145f88c668b39
* | Trivial: Update pypi url to new url3.36.1Tovin Seven2018-04-201-1/+1
| | | | | | | | | | | | | | | | | | Pypi url changed from [1] to [2] [1] https://pypi.python.org/pypi/<package> [2] https://pypi.org/project/<package> Change-Id: I53bfd078ba2a47a5744c2ebf1354d06ad6438544
* | eventletutils: Fix behavior discrepency when reusing EventsJohn Eckersberg2018-04-112-2/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The threading.Event object allows calling set() multiple times, but the eventlet.Event object only permits send() to be called once, before a reset() is required to reuse the Event. Calling eventletutils.Event.set() multiple times triggers an AssertionError from eventlet.Event. This change resets the underlying eventlet.Event if the set() method had already been called previously, and ensures the eventletutils.Event behaves the same as the threading.Event. Change-Id: If761b237266bbfe7e65c56e152074b5d1ccac74b
* | Imported Translations from ZanataOpenStack Proposal Bot2018-03-011-3/+3
| | | | | | | | | | | | | | For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: If10933e0bbc543f4a9ed686b96e13e43b3d8a296
* | Merge "Clean imports in code"Zuul2018-02-091-15/+12
|\ \
| * | Clean imports in codeNguyen Hung Phuong2018-01-301-15/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | In some part in the code we import objects. In the Openstack style guidelines they recommend to import only modules. https://docs.openstack.org/hacking/0.10.3/ Change-Id: Ie0151c2878c6a0ae09b7efcd5ccc7b31c622c0e7
* | | Imported Translations from ZanataOpenStack Proposal Bot2018-02-081-4/+17
| | | | | | | | | | | | | | | | | | | | | For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: Ic58353cda88124d10f320e62d19034a6c1278edd
* | | Merge "Document specs_matcher.py functions"Zuul2018-02-072-4/+78
|\ \ \
| * | | Document specs_matcher.py functionsJohn L. Villalovos2018-01-302-4/+78
| |/ / | | | | | | | | | | | | | | | | | | | | | Add some documentation to the specs_matcher.py functions. Add some additional unit tests. Change-Id: Ie399388e96d7a93fcb7e813b4b4da9ebd5d99688
* | | Fix breaking unit tests due to iso8601 changesJohn L. Villalovos2018-02-062-30/+31
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The move from iso8601===0.1.11 to iso8601===0.1.12 broke unit tests in oslo.utils. iso8601 used to do: from datetime import datetime But now they call datetime.datetime(): import datetime datetime.datetime() Unfortunately the unit tests that mocked datetime.datetime() are now mocking the one in iso8601. This causes a failure in the unit tests. Fix this by using the 'wraps' argument to mock. So that the calls will get passed through to datetime.datetime. Also changed to using the decorator style of mock. In addition Python 3 unit tests were broken due to changing how the UTC time zone is represented from 'UTC' to 'UTC+00:00'. Closes-Bug: #1747575 Closes-Bug: #1744160 Change-Id: Ia80ffb5e571cc5366bef2bc1a32c457a3c16843d
* | Replace 'assertFalse(a in b)' with 'assertNotIn(a, b)'sunyandi2018-01-191-1/+1
| | | | | | | | | | | | Trivial fix. Change-Id: Ie89f2bd03822cb76ee09d1d91284e7d356f5e5c5
* | Merge "Add a mixed mode parser to string_to_bytes"3.35.0Zuul2018-01-132-2/+35
|\ \
| * | Add a mixed mode parser to string_to_bytesIan Wienand2017-09-132-2/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This proposes a "mixed" parsing type that interprets the IEC-ish "i" units as power-of-two and the SI-style "non-i" units as power-of-ten. This is a round-about way of saying it is basically making it the same as the way GNU parted interprets such strings [1]. While I'd be the first to admit it's a bit ugly, I think it could serve a role for people who have to interpret such strings for compatability reasons. In particuarly within diskimage-builder, we've accepted these strings for partition creation (being modeled on parted) and I'd certainly like to not have our own unit parser if we can help it. [1] https://www.gnu.org/software/parted/manual/html_node/unit.html Change-Id: I772bcb6651484d68afabd1965e5db033439c6257
* | | Merge "Add missing information in docstring of validate_integer"3.34.0Zuul2018-01-071-0/+3
|\ \ \
| * | | Add missing information in docstring of validate_integerChangBo Guo(gcb)2017-12-181-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | Add versionadded and raises information in docstring Change-Id: Ia5929dc81cfed927a7b5dd809c13d3ae7efdf699
* | | | Merge "Add method to escape ipv6 ip addresses"Zuul2018-01-052-0/+18
|\ \ \ \
| * | | | Add method to escape ipv6 ip addressesDerek Higgins2017-08-182-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | IPv6 addresses are commonly enclosed in square brackets in resource identifiers to allow them to be distinguished from port numbers. I've come accross the need of this in multiple places zaqar https://review.openstack.org/#/c/495279 tripleo https://review.openstack.org/#/c/494440 ironic https://review.openstack.org/#/c/411809 ironic-python-agent https://review.openstack.org/#/c/411817 Change-Id: Icdcf681c1d71a09b88b029f80d13cd5015dacd56
View Lorry Controller Status