diff options
author | Lorin Hochstein <lorin@nimbisservices.com> | 2012-03-02 09:39:36 -0500 |
---|---|---|
committer | Lorin Hochstein <lorin@nimbisservices.com> | 2012-03-03 20:56:24 -0500 |
commit | d304d828d0422025af93bbb76b76843f5886a45f (patch) | |
tree | d401bcd6a912a86428cd4d2ab5a988539d08e72a | |
parent | f3a7149a85f78843422a1bf2c822ff8f47f08d7b (diff) | |
download | nova-d304d828d0422025af93bbb76b76843f5886a45f.tar.gz |
Added docs on MySQL queries blocking main thread.
Added to the RST docs to describe how queries to MySQL will block a nova
service.
Change-Id: I4ae209e2827978cc5469162d386197f2e46b342d
-rw-r--r-- | .mailmap | 1 | ||||
-rw-r--r-- | Authors | 2 | ||||
-rw-r--r-- | doc/source/devref/threading.rst | 46 |
3 files changed, 42 insertions, 7 deletions
@@ -38,6 +38,7 @@ <josh@jk0.org> <josh.kearney@rackspace.com> <justin@fathomdb.com> <justinsb@justinsb-desktop> <justin@fathomdb.com> <superstack@superstack.org> +<lorin@nimbisservices.com> <lorin@isi.edu> <masumotok@nttdata.co.jp> Masumoto<masumotok@nttdata.co.jp> <masumotok@nttdata.co.jp> <root@openstack2-api> <matt.dietz@rackspace.com> <matthewdietz@Matthew-Dietzs-MacBook-Pro.local> @@ -112,7 +112,7 @@ Koji Iida <iida.koji@lab.ntt.co.jp> Liam Kelleher <liam.kelleher@hp.com> Likitha Shetty <likitha.shetty@citrix.com> Loganathan Parthipan <parthipan@hp.com> -Lorin Hochstein <lorin@isi.edu> +Lorin Hochstein <lorin@nimbisservices.com> Lvov Maxim <usrleon@gmail.com> Mandell Degerness <mdegerne@gmail.com> Mark McLoughlin <markmc@redhat.com> diff --git a/doc/source/devref/threading.rst b/doc/source/devref/threading.rst index e499f47e1f..1c8eb5b954 100644 --- a/doc/source/devref/threading.rst +++ b/doc/source/devref/threading.rst @@ -1,17 +1,51 @@ Threading model =============== -All OpenStack services use *green thread* model of threading, implemented -through using the Python `eventlet <http://eventlet.net/>`_ and +All OpenStack services use *green thread* model of threading, implemented +through using the Python `eventlet <http://eventlet.net/>`_ and `greenlet <http://packages.python.org/greenlet/>`_ libraries. -Green threads use a cooperative model of threading: thread context -switches can only occur when specific eventlet or greenlet library calls are -made (e.g., sleep, certain I/O calls). From the operating system's point of +Green threads use a cooperative model of threading: thread context +switches can only occur when specific eventlet or greenlet library calls are +made (e.g., sleep, certain I/O calls). From the operating system's point of view, each OpenStack service runs in a single thread. The use of green threads reduces the likelihood of race conditions, but does -not completely eliminate them. In some cases, you may need to use the +not completely eliminate them. In some cases, you may need to use the ``@utils.synchronized(...)`` decorator to avoid races. +In addition, since there is only one operating system thread, a call that +blocks that main thread will block the entire process. +Yielding the thread in long-running tasks +----------------------------------------- +If a code path takes a long time to execute and does not contain any methods +that trigger an eventlet context switch, the long-running thread will block +any pending threads. + +This scenario can be avoided by adding calls to the eventlet sleep method +in the long-running code path. The sleep call will trigger a context switch +if there are pending threads, and using an argument of 0 will avoid introducing +delays in the case that there is only a single green thread:: + + from eventlet import greenthread + ... + greenthread.sleep(0) + + +MySQL access and eventlet +------------------------- +Queries to the MySQL database will block the main thread of a service. This is +because OpenStack services use an external C library for accessing the MySQL +database. Since eventlet cannot use monkey-patching to intercept blocking +calls in a C library, the resulting database query blocks the thread. + +The Diablo release contained a thread-pooling implementation that did not +block, but this implementation resulted in a `bug`_ and was removed. + +See this `mailing list thread`_ for a discussion of this issue, including +a discussion of the `impact on performance`_. + +.. _bug: https://bugs.launchpad.net/nova/+bug/838581 +.. _mailing list thread: https://lists.launchpad.net/openstack/msg08118.html +.. _impact on performance: https://lists.launchpad.net/openstack/msg08217.html |