| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
| |
The option was deprecated a long time ago, and will be removed in one of
the next library releases, which will render neutron broken if we keep
using the option.
More details:
http://lists.openstack.org/pipermail/openstack-dev/2016-May/095166.html
Closes-Bug: #1586066
Change-Id: I884b4cc3ed04e4b5489e265c146666e04eb1bc27
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
1) It checks that person who wants to trace is trusted and knows
secret HMAC key.
2) It starts tracing in case of proper trace headers
and adds first wsgi trace point, with info about HTTP request
* Add initialization of osprofiler at start of service
Currently that includes oslo.messaging notifer instance creation
to send Ceilometer backend notifications.
Neutron client change: Ic11796889075b2a0e589b70398fc4d4ed6f3ef7c
Co-authored-by: Ryan Moats <rmoats@us.ibm.com>
Depends-On: I5102eb46a7a377eca31375a0d64951ba1fdd035d
Closes-Bug: #1335640
DocImpact Add devref and operator documentation on how to use this
APIImpact
Change-Id: I7fa2ad57dc5763ce72cba6945ebcadef2188e8bd
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refactoring neutron configuration options for service to be in
neutron/conf. This would allow centralization of all configuration
options and provide an easy way to import.
Co-Authored-By: Dariusz Smigiel <smigiel.dariusz@gmail.com>
Partial-Bug: #1563069
Change-Id: Iff140c11765199536a8fd8066ad3063ed92355ae
|
|
|
|
|
|
|
|
|
|
| |
I think this needs a little more thought. This broke OVN at least and
I don't think that's good. We need to figure out how to be compatible
with existing plugins, even out of tree ones.
This reverts commit 1cafff087194711399ad2a85a9f394f7204d7bdd.
Change-Id: Ie087fb11213cc85911483c2d32c463fa9c973e54
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Forking a process when multiple threads are running is an unsafe
operation and could cause a lot of problems because only current
thread will continue working in child thread. Any locked by other
thread resource will remain locked forever.
We faced with this problem during oslo.messaging development and
added workaround to hide this problem:
https://review.openstack.org/#/c/274255/
I tried to fix this problem in oslo.service:
https://review.openstack.org/#/c/270832/
but oslo folks said that this fix is ugly and it is wrong way to add
workarounds to common libraries because projects use them incorrectly.
I think that is fair.
So this patch fixes incorrect usage of oslo libraries. In this patch
I extended functionality of NeutronWorker and add there
`worker_process_count` parameter which determines how many processes
should be spawned for this worker. If `worker_process_count` = 0 - don't
create process and spawn thread in scope of current process for worker
Then I moved all background tasks to workers and return them by
`get_workers` method. start_plugin_workers collects plugin's workers
using `get_workers` method and starts in ProcessLauncher first workers
with `worker_process_count` > 0 and only after this starts threaded
workers by simple Launcher
Closes-bug: #1569404
Change-Id: I0544f1d47ae53d572adda872847a56fa0b202d2e
|
|
|
|
|
|
|
|
|
|
|
| |
When neutron starts an rpc worker, it checks if the plugin has the method
"start_rpc_listeners". Since most plugins inherit from a base class, and
that base class implements the start_rpc_listeners method and raises
NotImplementedError, the rpc worker will attempt to call that method.
It should just catch the NotImplementedError and continue on.
Change-Id: Ie1830b6140acffffd0f283a0d8eefa52067f7650
Closes-Bug: 1551542
|
|
|
|
|
|
|
|
|
| |
Use the same wsgi server as the homegrown method dispatch
so we get support for api_workers and plugin workers when
using pecan.
Partially-Implements: wsgi-pecan-switch
Change-Id: I9fb942651d9b52b48e8cd80b0a6ab77ec37ea1e8
|
|
|
|
|
| |
Partial-Bug: #1520094
Change-Id: I874a4aa1d71d1f7034a1ff0b7450b419ef5c6864
|
|
|
|
|
|
|
|
|
|
|
|
| |
- This does NOT break other projects that rely on neutron.i18n,
as this change includes a debtcollector shim to maintain those
older entry points, until they can migrate.
- Also updates _i18n.py to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:
http://docs.openstack.org/developer/oslo.i18n/usage.html
Partially-Closes-Bug: #1519493
Change-Id: I1aa3a5fd837d9156da4643a367013c869ed8bf9d
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
By default spawn one additional rpc worker to process
state report queue.
State report queue will also be processed by regular
rpc workers, but in case these workers are busy with
processing heavy requests, state reports queue will
automatically be consumed by dedicated rpc workers.
This change applies to ML2 plugin only.
Other plugins should implement start_rpc_state_reports_listener
to enable additional rpc workers.
Change-Id: I5f8df6a478f7c82382049274b34b07109eeafbdb
Closes-Bug: #1505217
|
|
|
|
|
|
|
|
| |
Patch 5be613490deebf494f2ecbbcd8cb5dd0d6f5e1b3 erroneously removed this
line that is still needed which broke networking-ovn, at least.
Change-Id: I31981e905f892df474c57b45d3e70ebfbeeae1db
Signed-off-by: Russell Bryant <rbryant@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
This patch adds all RPC workers to consumers of service
plugins queues such as metering and l3-plugin.
This is important for DVR-enabled deployments with hundreds
of agents.
Change-Id: I6fea7f409c91b25d2c35b038d6100fdfa85d1905
Closes-Bug: #1498844
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are several cases where plugin initialization should be
handled after neutron-server forks API/RPC workers. For example,
starting a client connection to an SDN controller before forking
copies the fd of the socket to the child process, but then you have
multiple processes trying to read/write the same socket connection.
It is also useful for a plugin to be able to do something in only
one process, regardless of how many workers are forked. One example
would be handling syncing from an external system to the neutron
database.
This patch does 3 things:
1) Treats rpc_workers=0 as = 1. This simplifies the code for
handling notification that forking has completed. In the
existing code, calling the notification in the Worker object's
start() method would happen twice in the case where both api
and rpc workers were 0, despite there being only one process.
An earlier patch already changed the default api_workers to be
the number of processors.
2) Adds notification of forking via the callbacks mechanism.
Plugins can subscribe to resources.PROCESS, event.AFTER_CREATE
and do any post-fork initialization that needs to be done for
every spawned process.
3) Adds core/service plugin calls to get_workers() which defaults
to returning (). Plugins that need additional processes to spawn
should just return an iterable of NeutronWorkers that will be
spawned in their own process.
DocImpact
Closes-Bug: #1463129
Change-Id: Ib99954678c2b4f32f486b537979d446aafbea07b
|
|
|
|
|
|
|
|
| |
It looks like recent changes to oslo.messaging master are conflicting
with changes in neutron master with the way RPC services are started
when the rpc_workers value == 0.
Change-Id: Iea2197ad0ea9ceb9a2a850a9e03e53b4b39ca288
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch ensures log_opt_values is invoked only once, thus
avoiding annoying multiple dumps of options values.
Note that is rpc_workers is greater than 0 a second dump will
be logged. This patch does not address this as REST and RPC
servers are going to be separated as a part of the feature/pecan
work.
Closes-Bug: #1477975
Change-Id: Ia5dcb609241de6ad30d9831c5fb98a9e2be6ad7f
|
|
|
|
|
|
|
|
|
|
| |
oslo.service has graduated, so neutron should consume it.
Closes-Bug: #1466851
Depends-On: Ie0fd63f969f954029c3c3cf31337fbe38f59331a
Depends-On: I2093b37d411df9a26958fa50ff523c258bbe06ec
Depends-On: I4823d344878fc97e66ddd8fdae25c13a34dede40
Change-Id: I0155b3d8b72f6d031bf6f855488f80acebfc25d4
|
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit 12a564cf03e612dda36df26df8d28dfc75f1af6e.
We should re-enable this feature on a controlled basis so that we can
flush out any outstanding issue we may have.
Related-bug: #1432189
Change-Id: I2cfd93fdb032b461022b729347390ff8636ccdeb
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
All launchers implemented in common.service require each service to
implement reset method because it is called in case a process
receives a SIGHUP.
This change adds the reset method to neutron.service.RpcWorker and
neutron.wsgi.WorkerService which are used to wrap rpc and api
workers correspondingly.
Now neutron-server running in multiprocess mode (api_workers > 0 and
rpc_workers > 0) and metadata agent don't die on receiving SIGHUP and support
reloading policy_path and logging options in config.
Note that reset is called only in case a service is running in daemon mode.
Other changes made in the scope of this patch that need to be mentioned:
* Don't empty self._servers list in RpcWorker's stop method
When a service is restarted all services are gracefully shutdowned,
resetted and started again (see openstack.common.service code).
As graceful shutdown implies calling service.stop() and then
service.wait() we don't want to clean self._servers list because
it would be impossible to wait for them to stop processing
requests and cleaning up their resources.
Otherwise, this would lead to problems with rpc after starting
the rpc server again.
* Create a duplicate socket each time WorkerService starts
When api worker is stopped it kills the eventlet wsgi server
which internally closes the wsgi server socket object. This server
socket object becomes not usable which leads to "Bad file
descriptor" errors on service restart.
Added functional and unit tests.
DocImpact
Partial-Bug: #1276694
Change-Id: I75b00946b7cae891c6eb192e853118e7d49e4a24
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently pool dispose() is done after os.fork, but this will
produce shared DB connections in child processes which may lead
to DB errors.
Move pool dispose() before os.fork. This will remove all existing
connections in the parent process and child processes will create
their own new ones.
Change-Id: Ie36417a64f0eb39b53dad61517f834aec37bacfb
Closes-Bug: 1458718
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
All launchers implemented in common.service require each service to
implement reset method because it is called in case a process
receives a SIGHUP.
This change adds the reset method to neutron.service.Service class
which is used to start dhcp and l3 agents.
Now dhcp and l3 agents don't die on receiving SIGHUP and support
reloading policy_path and logging options in config.
Partial-Bug: #1276694
Change-Id: I96010e44928a665bea546865b2c81bde4ed0adf2
|
|/
|
|
|
|
|
|
|
|
|
| |
Avoid calling neutron.db.api.get_engine().pool.dispose() if
an engine facade has not yet been created since there won't
be any connections to get rid of. Calling it on services
that do not use the DB (e.g. agents) unnecessarily creates
a database connection engine that will never be used.
Change-Id: I3dbad1bef5da7b3765898e7d539b4d119b89e73a
Closes-Bug: #1433536
|
|
|
|
|
|
|
|
|
| |
This reverts commit c65175811baf29e75556045e3b44ab948e02a9c3.
This is in relation to:
https://bugs.launchpad.net/neutron/+bug/1432065
Change-Id: I97562840ff57f43958c1e3e56aa41219bd01c8c5
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The current default for api_workers is 0. This patch changes it to
use the number of CPUs available, which is the default that will
provide best performance. This is consistent with some other
OpenStack service (Nova and Swift, at least, perhaps others).
Note that devstack is updated in another patch to ensure multiple API
workers is being tested for Neutron the same way it's tested for
several other services:
https://review.openstack.org/140482
Co-Authored-By: Eugene Nikanorov <enikanorov@mirantis.com>
DocImpact
UpgradeImpact
Change-Id: I4d76baf68b014fc80693f53874734783089abda2
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It's mostly a matter of changing imports to a new location.
Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
from its thread local store and not local.store from incubator
* don't store context at local.store now that there is no code that
would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
not exist
* WritableLogger is now located in oslo_log.loggers
Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).
Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.
Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.
Closes-Bug: #1425013
Change-Id: I310e059a815377579de6bb2aa204de168e72571e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Oslo project decided to move away from using oslo.* namespace for all their
libraries [1], so we should migrate to new import path.
This patch applies new paths for:
- oslo.config
- oslo.db
- oslo.i18n
- oslo.messaging
- oslo.middleware
- oslo.rootwrap
- oslo.serialization
- oslo.utils
Added hacking check to enforce new import paths for all oslo libraries.
Updated setup.cfg entry points.
We'll cleanup old imports from oslo-incubator modules on demand or
if/when oslo officially deprecates old namespace in one of the next
cycles.
[1]: https://blueprints.launchpad.net/oslo-incubator/+spec/drop-namespace-packages
Depends-On: https://review.openstack.org/#/c/147248/
Depends-On: https://review.openstack.org/#/c/152292/
Depends-On: https://review.openstack.org/#/c/147240/
Closes-Bug: #1409733
Change-Id: If0dce29a0980206ace9866112be529436194d47e
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This change enforces log hints use and removes debug level log
translation, modifications are validated through a hacking rule
and the change respects logging guidelines.
Change-Id: Ia4e6659b8b59732d88cd603b0e6c630fad37aae5
Closes-bug: #1320867
|
|/
|
|
|
|
| |
The docstring for topic is not correct.
Change-Id: I6174453eb1c41c398ecffbab59c45b0678637672
|
|
|
|
|
|
|
|
|
|
|
|
| |
The following modules are removed:
- excutils,
- importutils,
- network_utils,
- strutils,
- timeutils.
Closes-Bug: #1385355
Change-Id: I1f34f17f5dbf37032584008f27e65d4dc4d475f4
|
|
|
|
|
|
|
|
| |
MessageHandlingServer has no kill() method. In case neutron server using
rpc workers and is stopped, wrong method is called.
Change-Id: Id4a11f3dca070e684fb9e079139ac09792bc7aad
Closes-Bug: 1387053
|
|
|
|
|
|
|
|
|
| |
Refactor a few ugly aspects of the multiple API worker patch to make
way for multiple rpc workers. This came up as I was trying to add
multiple RPC workers using similar patterns and remembering that some
things were left in a rather awkward state.
Change-Id: I549db67af4af6a2df80e12cf233109dda5213c47
|
|\ |
|
| |
| |
| |
| |
| |
| |
| | |
Prefer single style of throwing instances and not classes.
Change-Id: If159afcd092de5381309cbe217d64145ed5b45b4
Closes-Bug: #1364392
|
|/
|
|
|
|
|
|
| |
The cfg.CFG argument is ignored and misleading. This change removes it
and updates all callers.
Closes-Bug: #1355565
Change-Id: I2fcece85d1abe848e5c01125cfb62c02f2dcbc86
|
|
|
|
|
|
|
|
|
| |
Most of this code will probably stay with us for quite some time, so
let's make things easier and consider them as our way of doing RPC.
blueprint oslo-messaging
Change-Id: Iaf353b23f9c54b82d1e02a6bd5a5960cec827c88
|
|
|
|
|
|
|
|
|
| |
Now that we explicitly start multiple RPC servers, renamed the method to
reflect behaviour.
blueprint oslo-messaging
Change-Id: I7c51cc1b44b4b4c8899255b9bedce1792c38634e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Now that all preparations are done, actually port the code to use
oslo.messaging. This patch does as little as possible. Follow up patches
that refactor and cleanup the code and configuration files, will be
merged later. The reason for this is to make the patch as slim as
possible, to make review process more smooth and concentrated.
Details:
* neutron/common/rpc.py:
- added init() and cleanup() to set global RPC layer state.
- added utility functions: get_server(), get_client(), get_notifier()
that wrap up oslo.messaging API a bit, enforcing eventlet executor
and setting serializer, among other things.
- removed PluginRpcDispatcher, instead introduced PluginRpcSerializer
to use as a default serializer for API callbacks.
* neutron/common/rpc_compat.py:
- emulated incubator RPC layer behaviour thru previously introduced
stub classes (RpcCallback, RpcProxy, ...) using new oslo.messaging
API.
- switched to using new oslo.messaging exception types.
* neutron/service.py:
- expect multiple RPC listeners that are of MessageHandlingServer
type, not GreenThread.
* neutron/common/config.py:
- initialize RPC layer in init()
* setup.cfg:
- added entry points for old notifier drivers to retain backward
compatibility.
* neutron/tests/...:
- introduced fake_notifier to replace impl_fake.
- faked out consume_in_thread() to avoid starting RPC listeners when
running unit tests.
- used 'fake' transport driver.
- made sure neutron.test.* exceptions are caught.
- initialize and clean up RPC layer for each test case.
* Ported all affected code from using neutron.openstack.common.notifier
API to oslo.messaging.Notifier.
* rpc.set_defaults() was renamed to rpc.set_transport_defaults()
* other changes not worth mentioning here.
blueprint oslo-messaging
DocImpact
Change-Id: I5a91c34df6e300f2dc46217b1b16352fcc3039fc
|
|
|
|
|
|
| |
blueprint oslo-messaging
Change-Id: Ie48de6d3636d6404316f19d73c7e8453298ecf14
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Synced from commit 5fb343faee1442921e2d610b1a5222e67418c4cd.
Due to changes in oslo.db API, this sync requires a bit more
work on Neutron side.
Sync includes the following commits:
5b7e61c Dispose db connections pool on disconnect
d1988b9 Set sql_mode callback on connect instead of checkout
a1a8280 Fix excessive logging from db.sqlalchemy.session
9933bdd Get mysql_sql_mode parameter from config
96a2217 Prevent incorrect usage of _wrap_db_error()
20a7510 Add from_config() method to EngineFacade
fea119e Drop special case for MySQL traditional mode, update unit tests
dda24eb Introduce mysql_sql_mode option, remove old warning
0b5af67 Introduce a method to set any MySQL session SQL mode
8dccc7b Handle ibm_db_sa DBDuplicateEntry integrity errors
5b9e9f4 Fix doc build errors in db.sqlalchemy
ac84a40 Update log translation domains
86707cd Remove None for dict.get()
0545121 Fix duplicating of SQL queries in logs
fcf517d Update oslo log messages with translation domains
630d395 Don't use cfg.CONF in oslo.db
ce69e7f Don't store engine instances in oslo.db
Change-Id: I0e1d86878d3eb924b01e04dced0f90b4e57757d8
|
|
|
|
|
| |
Change-Id: If5e07fa4529430ba90d9c7dd81d28ceeee00ebe0
Partial-Bug: #1291032
|
|
|
|
|
|
|
|
|
| |
Metaplugin needs a fix to support multiple RPC workers properly
because a plugin which supports multiple RPC workers cannot
initialize RPC connections at plugin initialization.
Closes-Bug: #1300570
Change-Id: I584f70abb8969054cd4edc8f914d00f6be930bab
|
|
|
|
|
|
|
| |
Goodbye Quantum!
Change-Id: Idbd0384a892beaff3a937444f04cfc433cb805eb
Closes-Bug:1299046
|
|
|
|
|
|
|
| |
blueprint multiple-rpc-workers
Co-Authored-By: Terry Wilson<twilson@redhat.com>
Change-Id: I51f2a52add6c11af905e6f1e6e45d31731ebbb5d
|
|
|
|
|
|
|
|
|
|
| |
In the process __init__.py involved have been emptied;
vim modelines and author lines have been dropped from
the license headers affected by the change as well.
Partial-implements blueprint nicira-plugin-renaming
Change-Id: I9c535b59bb2e83f275816472e0e32b41444d3bbe
|
|
|
|
|
|
|
| |
This commit fixes neutron core and common codes.
Partial-Bug: #1279813
Change-Id: Id109924c7ff9b4f0d74c23665016a54bfd7dff77
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This change to the WSGI code uses openstack.common.service to create
multiple worker processes to handle API load. The main process will
start up a configurable (workers=??) number of child processes which
will all listen on the bind port. The main process becomes the parent
and manages the children. The parent is not a worker.
Backwards compatibility is preserved by setting api_workers to 0, the
default. In this case, no separate worker processes are spawned and
the worker threads run in the main process.
Implement blueprint multi-workers-for-api-server
Change-Id: Iffa76041d0055840ccca852814b0e71f17a950ac
|
|/
|
|
|
|
|
|
|
|
| |
Raise log level at the first load attempt. If there is a
fundamental problem with the environment, a Lookup error
is not helpful in troubleshooting the problem further.
Fix bug 1210236
Change-Id: I55aa11a0e5a37dbef3f0c1b55aa4c9c11bbf55d9
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Partial-Bug: #1217100
Using tools/check_i18n.py to scan source directory, and fix most of
the errors.
- Message internationalization
- First letter must be capital
- Using comma instead of percent in LOG.xxx
Note: all extension's description are not touched in this patch,
can be fixed after discussing.
Note: all nicira/check_nvp_config.py print messages are not fixed.
Change-Id: I79ef06fd42f6780beb5019c592662536c2a51864
|