summaryrefslogtreecommitdiff
path: root/glance_store/common
Commit message (Collapse)AuthorAgeFilesLines
* Cinder: Add support to extend attached volumesRajat Dhasmana2023-01-181-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | While creating an image, if we want to extend the volume (to accomodate the image), we need to first detach the volume, perform the extend and attach it again. This is inefficient for backends that support extending attached volumes since we are performing 3 cinder operations (the attachment call includes 3 API calls which sum to a total of 5 API calls for 3 operations) instead of directly extending it which requires only 1 API call to cinder. The support for extending attached volumes was added in cinder microversion 3.42. This patch adds a new boolean config option ``cinder_do_extend_attached`` which allows operators to specify if the cinder backend they are using supports extending attached (in-use) volumes. By default this will be ``false``. Based on the parameter, we will perform the extend operation, online (if cinder_do_extend_attached is true) and offline otherwise. Spec: https://review.opendev.org/c/openstack/glance-specs/+/868901 Depends-On: https://review.opendev.org/c/openstack/glance/+/869021 Depends-On: https://review.opendev.org/c/openstack/cinder/+/869051 Change-Id: I5e70824e9abc5277ea25ba95704b358fe3686037
* Refactor/restructure glance cinder storewhoami-rajat2023-01-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is an effort to decouple some of the cinder backend specific code (like nfs, scaleio) from the generic logic. The purpose is to make the code modular and any change for a particular cinder backend should not affect the code path of other backends thereby reducing regression. This is also required for another use case of supporting extend of attached volumes added in [1]. Following are the major changes done in this patch: 1) Move cinder store to a new directory 'cinder' and rename 'cinder.py' to 'store.py' (similar to swift) 2) Create new files for nfs and scaleio backends for moving code specific to these backends into their own separate file. This also fixes one bug when using sparse files in nfs and we wait for file size to be equal to volume size (initially done for scaleio/powerflex backend) but this will never happen for nfs sparse files. See bug: 2000584 3) Move cinder tests to 'tests/unit/cinder' directory and add tests for base, nfs and scaleio files. 4) Modify/fix existing tests Closes-Bug: #2000584 [1] https://review.opendev.org/c/openstack/glance_store/+/868742 Depends-On: https://review.opendev.org/c/openstack/glance/+/869021 Change-Id: I26c272b6c503e98fbbafca411d3eec47283bd6fc
* Merge "Cinder: Correct exception logging during attach"Zuul2022-05-051-2/+2
|\
| * Cinder: Correct exception logging during attachwhoami-rajat2022-04-281-2/+2
| | | | | | | | | | | | | | | | | | | | There is a syntax error in the exception logging when attaching a volume fails in attachment state manager. This patch corrects it and adds a test to guard against similar changes in future. Closes-Bug: #1970698 Change-Id: I43c407046a49bb37631113e2ea65d05450f9365d
* | Correct retry interval during attach volumewhoami-rajat2022-04-181-1/+3
|/ | | | | | | | | | | | | | | | | | | | | | | | | | When we try to simultaneously attach same volume multiple times (multiattach), there are multiple attachments created, suppose attachA and attachB. If attachA marks the volume "reserved" then attachB can't proceed until the volume is in "in-use" or "available" state. We retry until we reach any of these states for which we use the retrying library. The retrying library defaults to 1 second retry[1] (5 times) which causes the original failure as the volume takes time to transition between "reserved" -> "in-use" state. This patch corrects it by adding an exponential retry time and max exponential retry i.e. 1: 2 ** 1 = 2 seconds 2: 2 ** 2 = 4 seconds 3: 2 ** 3 = 8 seconds 4: 2 ** 4 = 16 seconds (but max wait time is 10 seconds) 5: 2 ** 5 = 32 seconds (but max wait time is 10 seconds) [1] https://github.com/rholder/retrying/blob/master/retrying.py#L84 Closes-Bug: #1969373 Change-Id: I0094b044085d7f92b07ea86236de3b6efd7d67ea
* nit: Correct debug logwhoami-rajat2021-11-221-1/+1
| | | | | | Correct debug log when initializing _AttachmentStateManager. Change-Id: Ifa909a67c76b897f2ab4b1ee5926046bc7045d3d
* Add volume multiattach handlingwhoami-rajat2021-08-122-5/+281
| | | | | | | | | | | | | | | | We implemented cinder's new attachment API support with patch[1]. It was needed to add multiattach volume handling added with this patch. There is no special configuration change or user interference needed. If a volume is of a multiattach type, then it will be handled as a multiattach volume. [1] https://review.opendev.org/c/openstack/glance_store/+/782200 Implements: blueprint attachment-api-and-multiattach-support Closes-Bug: #1904546 Change-Id: Iffb825492a20fd877476acad05f817b399072f01
* Glance cinder nfs: Block creating qcow2 volumesRajat Dhasmana2021-07-281-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | There's an issue when cinder nfs is configured as glance backend and the image is > 1GB i.e. extend operation is performed on volume. Currently glance writes the qcow2 header in the raw volume and cinder nfs driver used to work on format autodetection which is changed to manual detection here[1] This fixes the extend volume problem for raw volumes, however, if cinder nfs is configured to create qcow2 volumes then we will run into the same problem. This patch blocks creating images when the nfs volume is qcow2. Refactoring work: 1) attachment_delete call is removed in nfs related code as it was already done in the finally block 2) handle_exceptions decorator is removed from volume delete method as it raises BackendException and incase when volume is not found (possibly deleted manually), the image deletion fails so we don't want to raise any error in this case [1] https://review.opendev.org/c/openstack/cinder/+/761152 Closes-Bug: #1901138 Change-Id: I8ce6f36f1cb4b0ed6bcc5f3869fab3bb64fe3390
* Add cinder's new attachment supportRajat Dhasmana2021-07-231-0/+194
| | | | | | | | | | | | | | | | | Cinder introduced new attachment API flow in microversion 3.27 (also attachment_complete added in mv 3.44 and support for passing mode added in mv 3.54) which provides a clean interface to interact with cinder for attachments and is also required for multiattach volume support (Related future work). Nova uses it since a long time and is proven to be stable, this patch implements the same for glance. The create volume and delete volume calls are also moved to cinder_utils file to use the generic exception handler and keep similar code together for consistency. Partially Implements: blueprint attachment-api-and-multiattach-support Change-Id: I2758ed1d5b8e0981faa3eff6f83e1ce5975a01d2
* Replace md5 with oslo versionAde Lee2020-11-111-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | md5 is not an approved algorithm in FIPS mode, and trying to instantiate a hashlib.md5() will fail when the system is running in FIPS mode. md5 is allowed when in a non-security context. There is a plan to add a keyword parameter (usedforsecurity) to hashlib.md5() to annotate whether or not the instance is being used in a security context. In the case where it is not, the instantiation of md5 will be allowed. See https://bugs.python.org/issue9216 for more details. Some downstream python versions already support this parameter. To support these versions, a new encapsulation of md5() has been added to oslo_utils. See https://review.opendev.org/#/c/750031/ This patch is to replace the instances of hashlib.md5() with this new encapsulation, adding an annotation indicating whether the usage is a security context or not. It looks like the uses of the md5 are primarily for checksums and generation of etags. With this patch, all the unit and functional tests appear to pass on a FIPS enabled system. Change-Id: I0603ba217d6dc19f5c9f73c60c7b365efd28d30b Depends-On: https://review.opendev.org/#/c/760160
* [Trivial]Add missing white space between wordszhufl2020-07-281-1/+1
| | | | | | This is to add missing white space between words in log message. Change-Id: Ie166d3d29b10452238f6d3b325bb8c2f36b5fae1
* Add lock per share for cinder nfs mount/umountwhoami-rajat2020-05-012-0/+370
| | | | | | | | | | | | | | With cinder configured as glance store and nfs as cinder backend, if we create multiple images concurrently into the same share, there might be race conditions. This patch ensures a lock per export to avoid the race conditions. This patch also introduces a reference counting mechamism for unmounting which will unmount the share if no active thread/process is using it. Closes-Bug: #1870289 Change-Id: I9197f64e29a0ae2e0a58186f1a70aa134f7f1db6
* Python3: Fix return type on CooperativeReader.readCyril Roelandt2019-04-121-1/+1
| | | | | | | | CooperativeReader.read() should return bytes. A similar change was merged in Glance. Change-Id: Iee3b49235be2af64eac3933e41c7e44fc1a564f4 Closes-Bug: #1818915
* Use oslo_utils.encodeutils.exception_to_unicode()Victor Stinner2016-01-211-16/+0
| | | | | | | | | | | | | | | | | | Replace glance_store.common.utils.exception_to_str() with encodeutils.exception_to_unicode(). exception_to_unicode() tries more options to get a nice error message as Unicode from an exception object. As exception_to_str(), it catches and handles exceptions. Remove glance_store.common.utils.exception_to_str() and related unit tests. The function is not part of the store API and Glance does not use it. glance already uses exception_to_unicode(): see change I86008c8adc0c5664f96573c1015cc15e2d06e3e2. By the way, glance did not use exception_to_str() from glance_store but had its own copy of the function (which was removed by the mentioned change). Change-Id: I52cab2e773c257e36d36290f6060811f83f18bb0
* Remove unnecessary auth moduleMike Fedosin2016-01-191-293/+0
| | | | | | | | | | | | This commit removes common/auth.py file and changes the way how endpoint for MultiTenant store is found. Without having custom implementation it's better to use an unified method 'url_for' that service_catalog provides. Co-Authored-By: kairat_kushaev <kkushaev@mirantis.com> Change-Id: If8d07e378140a940317db1259a58477d7809948e
* Remove useless config.py fileMike Fedosin2015-11-181-220/+0
| | | | | | The change removes unused glance_store.common.config module Change-Id: Ie47901cb4161f5fc87c4ee1ceddef56d51ce78dc
* remove default=None for config optionsShuquan Huang2015-11-091-1/+1
| | | | | | | In the cfg module default=None is set as the default value. Change-Id: If4ae891bf5d098bb8b33804dcd5d3fb9826c594d Closes-bug: #1323975
* Activate pep8 check that _ is importedTHOMAS J. COCOZZELLO2015-10-053-0/+4
| | | | | | | | | | | Remove the specification in tox.ini that _ is a builtin so that it will no longer assume that _ does not need to be imported. This helps ensure that the _ from i18n is used. Activating this check did flag violations and they were fixed in this patch. Change-Id: Ia7e4c75b2126a67784683d1c22f0e24bc44113a4
* '_' is used by i18nTom Cocozzello2015-10-021-1/+1
| | | | | | | | | '_' is well know to be used for a variable that is not in use. But since gettext defines '_' for translations this must be changed to allow i18n to properly declare the '_'. Change-Id: Id9f9717999930add868247b4c1a50e1be4506e2d Closes-Bug:1502332
* Use six.moves to fix imports on Python 3Victor Stinner2015-06-101-3/+3
| | | | | | | | | | | | | | | | | | | | | * Use six.moves to get builtins, configparser, http_client and urllib modules * Replace StringIO.StringIO with six.StringIO * Replace reload(mod) with moves.reload_module(mod) Patch generated by the six_moves, stringio and urllib operations of the sixer tool version 0.4: https://pypi.python.org/pypi/sixer Manual changes: * test_vmware_store: add _mock_http_connection() helper method to factorize code mocking http_client.HTTPConnection * Reformat some lines to respect the constraint of 80 columns * Replace reload() with imp.reload_module() * Replace httplib with six.moves.http_client in mock.patch() Change-Id: I5a065c0778436a12db90ac3b929fa840f3e86665
* Merge "Port exception_to_str() to Python 3"Jenkins2015-06-101-1/+3
|\
| * Port exception_to_str() to Python 3Victor Stinner2015-05-201-1/+3
| | | | | | | | | | | | | | | | | | | | On Python 3, exception messages are unicode strings. There is no need to decode them. Skip the test on the "ignore" error handler on Python 3, it's specific to Python 2. Change-Id: Ieec9e1e1f7a0fc901271fd74eca357fe1cf6a975
* | Fix Python 3 issuesVictor Stinner2015-05-201-1/+1
|/ | | | | | | | | | | | | | | | | | * Replace it.next() with next(it) * Replace dict.iteritems() with six.iteritems(dict) * Replace unicode with six.text_type * Replace dict(d1.items() + {'Cookie': cookie}.items()) with: d3=dict(d1); d3['Cookie'] = cookie * Replace map(func, parts) with [func(part) for part in parts] * Open file in binary mode (not in text mode) to load JSON using jsonutils.load() * Avoid using dict.keys()[0] or dict.values()[0]: on Python 3, keys() and values() methods of dictionaries return iterators, not lists For more information on Python 3, see: https://wiki.openstack.org/wiki/Python3 Change-Id: Ib09a3367d8dcb664911a0b187da9e363a96727fd
* Update oslo librariesCindy Pallares2015-03-191-1/+1
| | | | Change-Id: I09ab6d5ceea5d0f7e031be100271ed3e2fc3d316
* Unify using six.moves.range rename everywhereErno Kuvaja2015-02-251-0/+2
| | | | | | | | | Mainly to improve consistency, use range() from six.moves renames across glance. Behaves consistently like py2 xrange() and py3 range(). Change-Id: I542fcb2a762cf4b34c30fc796854c6fcf305d798
* Move from oslo.config to oslo_configLouis Taylor2015-01-121-1/+1
| | | | | | | | | | oslo_config was moved out of the oslo namespace in oslo.config>=1.6.0. This updates the required version of oslo.config. Related-to: blueprint drop-namespace-packages Change-Id: I18f86c4f2838a14de344ef7887353ca0b5f1a1dd
* Move from oslo.utils to oslo_utilsLouis Taylor2015-01-091-1/+1
| | | | | | | | oslo_utils has been moved out of the oslo namespace. bp drop-namespace-packages Change-Id: Id4ccf076983592026d35407107e79cf1575248ce
* Switch to using oslo.utilsZhi Yan Liu2014-09-291-3/+2
| | | | | | | | This change switched glance_store to using oslo.utils. It is not good to add dependency on oslo-incubator if we can avoid it. Change-Id: I9efe3d57e0ce64296c7a75838bd2ee19249f0fa8 Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
* Use oslo.serializationOleksii Chuprykov2014-09-211-1/+1
| | | | | | | | Module `jsonutils` from common code was graduated to oslo.serialization, so it would be great if we reuse this library. Remove unused log.py and fileutils.py Change-Id: Ie7d80df501296b4b06da8c1cc155bcc4b64bfe28
* Merge "Adding common.utils.exception_to_str() to avoid encoding issue"Jenkins2014-09-192-1/+17
|\
| * Adding common.utils.exception_to_str() to avoid encoding issueZhi Yan Liu2014-09-192-1/+17
| | | | | | | | | | | | | | | | | | | | 1. Ported common.utils.exception_to_str() back to glance_store. 2. Used exception_to_str() to consistently cast the exception to avoid potential encoding issue. Change-Id: I398ea0e218ebb37b9041cfe82f18b11e614662bf Related-Id: I77478a8eeeefd85bf98138156429df53d0a4c079 Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
* | Portback part change of enabling F821 checkZhi Yan Liu2014-09-191-18/+20
|/ | | | | | Enable F821 and fix usage of undefined variables. Change-Id: I0d5104060410986df4b83a3e9e865365004d0313 Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
* Cleanup shebang in non-executable moduleHaikel Guemar2014-09-121-2/+0
| | | | Change-Id: Ic173e7c96b3d9d1d2f905fffb94e3d723f1dcdc3
* Rename glance.store to glance_store0.1.1Flavio Percoco2014-08-284-0/+647
Given the existing, known issues, with python namespaces, pip and setuptools, we've decided to rename glance.store into glance_store. Change-Id: I3f02ce78b3d64f34744e5116652bfbb4f3062cbf
View Lorry Controller Status