summaryrefslogtreecommitdiff
path: root/tools
Commit message (Collapse)AuthorAgeFilesLines
* tox cleanupsStephen Finucane2023-01-181-7/+7
| | | | | | | | | 'skip_basepython_conflicts' has been the cause of a couple of bugs in tox 4 and there is talk of it going away. Remove it and fix up a few other issues in the tox.ini file. Change-Id: Ic19c896af2ab0cf3570c43e8ceb8cba64fb45cdd Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* Implement get_rpc_client functionTobias Urdin2022-10-251-1/+1
| | | | | | | | | | | | | | | | | | | We already expose functions to handle the instantiation of classes such as RPCServer and RPCTransport but the same was never done for RPCClient so the API is inconsistent in its enforcement. This adds a get_rpc_client function that should be used instead of instatiating the RPCClient class directly to be more consistent. This also allows to handle more logic inside the function in the future such as if implementations for an async client is implemented, as investigation in [1] has shown. [1] https://review.opendev.org/c/openstack/oslo.messaging/+/858936 Change-Id: Ia4d1f0497b9e2728bde02f4ff05fdc175ddffe66
* Adding pre-commitHervé Beraud2020-09-221-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduced changes: - pre-commit config and rules - Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks. - Applying fixes for pre-commit compliance in all code. Also commit hash will be used instead of version tags in pre-commit to prevend arbitrary code from running in developer's machines. pre-commit will be used to: - trailing whitespace; - Replaces or checks mixed line ending (mixed-line-ending); - Forbid files which have a UTF-8 byte-order marker (check-byte-order-marker); - Checks that non-binary executables have a proper shebang (check-executables-have-shebangs); - Check for files that contain merge conflict strings (check-merge-conflict); - Check for debugger imports and py37+ breakpoint() calls in python source (debug-statements); - Attempts to load all yaml files to verify syntax (check-yaml); - Run flake8 checks (flake8) (local) For further details about tests please refer to: https://github.com/pre-commit/pre-commit-hooks Change-Id: Ibd0c3d64fdc5c293d9d676d33eab828d9fde971f Co-authored-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
* Simplify tools/test-setup.shAndreas Jaeger2020-07-011-22/+4
| | | | | | | | | | | | | | | | | | test-setup.sh is called after bindep is run in CI, and tox is not setup. Simplify the script so that it only does what's needed - install qdrouterd for Ubuntu from qpid/testing. Install the other packages using bindep, add test to amqp1 profile. Add amqp1 to bindep_profiles where needed for scenario tests, previously test-setup.sh added it everywhere. Some tests are skipped due to changes in recent versions of kombu. These will be unskipped when we come up with a proper fix. Change-Id: Ic3a9e2c873619670edfbf71022d593f3cb5f70f2 Related-Bug: #1885923
* Remove six usageAndreas Jaeger2020-05-111-7/+6
| | | | | | | | | Remove six, the python 2/3 compatibility library. It's not needed anymore since the repo is python3 only. Remove a now unneeded hacking test. Change-Id: I40522c4accb4aaf8115d11fee8b081e2d991cb4d
* Setup backend scenarios for functional testsAndy Smith2020-02-052-41/+69
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patchset introduces scenarios for the functional tests to unify the setup and configuration of alternate messaging backends for RPC and Notifications. The scenarios are defined to reduce redundancy in the testing of backends and to demonstrate functional correctness across driver combinations. Current driver support: rabbit - RPC, Notify amqp - RPC kafka - Notify RPC Notify --------- ---------- scenario01 rabbit rabbit scenario02 rabbit kafka scenario03 amqp rabbit scenario04 amqp kafka It is anticipated that additional scenarios will be defined as new drivers are introduced and/or new messaging intermediarites are supported. Note: The removal of python2 jobs are also included patch Change-Id: I0f86416623a0b718516147f0660b4df2b74cf867
* Merge "Consider the topic parameter as an array in client-notify"Zuul2019-04-291-1/+1
|\
| * Consider the topic parameter as an array in client-notifyGabriele2019-04-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Without this fix, the simulator splits the "topic" value in different strings, in notify-client case Instead of having one queue there are multi-queues. See [1] for more details Closes-Bug: #1824339 1- https://bugs.launchpad.net/oslo.messaging/+bug/1824339 Change-Id: I438c7279d584bdb00da18477a83d5e2a53c588c4
* | Update messaging intermediaries for amqp1 testsAndy Smith2019-03-111-191/+20
|/ | | | | | | | | | | | | | | | This patch messaging intermediaries used for the amqp1 driver test. The combination of the apache qpid-dispatch-router (qdrouterd) with an attached apache artemis broker is used in place of the qpidd broker. The qdrouterd will directly message oslo.messaging rpc communications and will link route oslo.messaging notify communications to the artemis broker (e.g. for persistence). This patch: * updates the dependencies * updates the test environment setup * uses the pifpaf drivers to create servers Change-Id: I9fb56450346fca84bc1573fa5e9b6fe81c14e33a
* Update hacking versionZhijunWei2019-01-211-2/+2
| | | | | | | | update the hacking to latest, duo to[1] [1]: https://github.com/openstack/oslo.messaging/blob/master/HACKING.rst Change-Id: I1ee7387272690f0de0d7b1937d7c2b0ee404063c
* update configuration for qdrouter v1.0.0Kenneth Giusti2018-03-201-1/+12
| | | | Change-Id: I20637c51d5ae4a96f1de9ea6e2a50891adaca52a
* Update kafka functional testAndrew Smith2017-12-161-1/+4
| | | | | | | | | | | | | | | | | | | | | This patch addresses a number of issues that prevented the functional tests from running. The functional tests now execute and can complete succesfully. At times, the test will fail (noticiably in CI) indicating an underlying issue with consumer interaction with the kafka server. It would be beneficial to merge this patch as it provides repeatability and visibility for driver-kafka server integration to facilitate additional debugging and testing. This patch: * removes use of deprecated get_transport * override consumer_group for each test * changed to synchronous send * update to kafka 1.0.0 server Depends-On: Ib552152e841a9fc0bffdcb7c3f7bc75613d0ed62 Change-Id: I7009a3b96ee250c177c10f5121eb73d908747a52
* Merge "Provide bindep_profile in openstack-tox job setup"Zuul2017-12-111-9/+7
|\
| * Provide bindep_profile in openstack-tox job setupAndrew Smith2017-11-291-9/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Oslo.messaging uses a single bindep.txt to define common and driver dependencies. The bindep_profile is used by the bindep role to select the driver specific packages. This patch: * Add bindep_profile var to jobs * Add kafka dependencies * update test-setup for use by amqp1 Depends-On: I6beb7b07a7dbc345e63e758ebbc86400e0ee16d3 Change-Id: Ib552152e841a9fc0bffdcb7c3f7bc75613d0ed62
* | Avoid tox_install.sh for constraints supportAndreas Jaeger2017-12-021-31/+0
|/ | | | | | | | | | | We do not need tox_install.sh, pip can handle constraints itself and install the project correctly. Thus update tox.ini and remove the now obsolete tools/tox_install.sh file. This follows https://review.openstack.org/#/c/508061 to remove tools/tox_install.sh. Change-Id: Ifaaf656effff20ef08214f111645a3b5fc8b4d28
* Move legacy zuulv3 tests into oslo.messaging repoKenneth Giusti2017-10-172-12/+211
| | | | | | Next step will be to re-write these for zuulv3 Change-Id: I2f6a8432e037f75d6abee3bc977933a310a4a2c7
* remove all kombu<4.0.0 workaroundsgord chung2017-03-301-3/+0
| | | | | | we lower cap at kombu>=4.0.0 so no need to support kombu3 stuff Change-Id: I4c3bc4de6026b089fffae7ae4d34f718d45cb972
* Fix simulator's use of Notifier - use 'topics' not 'topic'Kenneth Giusti2017-03-201-1/+1
| | | | | Change-Id: Id315418fcf517734b8136f8177c82951658fe794 Closes-Bug: 1673890
* support kombu4gord chung2017-02-101-0/+3
| | | | | | | | | | | | - kombu4 wraps recoverable errors as OperationalErrors rather than raising amqp errors - also, raise a recoverable error and redeclare if for some reason a message is double acknowledged... previously, this was hidden. - ensure socket is not none - use connect method to ensure connection Depends-On: I9f980b51901ac31599b9651633956ad2eea6a1ac Change-Id: I73958c8057353a2eefe1baaa7a41148193d507f7
* kafka: Remove testing hack for kafkaMehdi Abaakouk2017-01-261-6/+0
| | | | Change-Id: Ie304f1ed34156bfb60b9bc9acd4ad0aa5bad4657
* Moving driver to new kafka-python versionIlya Tyaptin2017-01-021-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | Currently Kafka driver for an oslo.messaging uses kafka-python==0.9.5 and mostly broken. This package version supports only low level Kafka producer and consumer API which are marked as deprecated now [1]. Using of these interfaces bring a big concern to the message processing, because current KafkaConsumer has not any consuming coordination. This fact causes a message duplication for the several consumers of one topic. This behavior is specific to Ceilometer and services which read and process notifications from other services. New version of kafka-python allows to use async thread safe message producers and coordinated consumers [1]. [1] http://kafka-python.readthedocs.io/en/master/changelog.html#feb-15-2016 The driver is currently experimental, python-kafka<1.0.0 API have major issue described above that can't make the oslo.messaging driver works, so we prefer having a working driver with a non-synced dependencies, that the reverse. Co-Authored-By: Mehdi Abaakouk <sileht@redhat.com> Change-Id: I29862ed7bf56b9d8878fa8e9fb1cbd9d643908a4
* [AMQP 1.0] Setup the amqp1 test environment on ubuntuKenneth Giusti2016-12-191-0/+1
| | | | | | | Fixes the amqp1 functional tests so they can be run on Xenial. Closes-Bug: 1643644 Change-Id: I3e3ee6779a60774ebffc1f45ae5564db934f8eaf
* tests: fix test-setup.shMehdi Abaakouk2016-12-151-1/+6
| | | | | | | tox doesn't have quiet option, so we have to create the virtualenv first and run bindep manually. Change-Id: I3ae76278391baa5cd80202f32ad88b1368b0f16d
* Add bindep.txt/test-setup.sh to prepare the systemMehdi Abaakouk2016-12-071-0/+36
| | | | Change-Id: I9ac3132c082a31020dc2bf2a778fb5c4c0836c4c
* Add Constraints supportTony Breeds2016-11-231-0/+30
| | | | | | | | | | | | Adding constraints support to libraries is slightly more complex than services as the libraries themselves are listed in upper-constraints.txt which leads to errors that you can't install a specific version and a constrained version. This change adds constraints support by also adding a helper script to edit the constraints to remove oslo.messaging. Change-Id: I8be883215f27abb58d15b85e8542cbdf32000bac
* Merge "[simulator] Fix transport_url usage"Jenkins2016-11-031-4/+0
|\
| * [simulator] Fix transport_url usageKirill Bespalov2016-10-191-4/+0
| | | | | | | | | | | | | | | | | | | | 1) Remove hardcoding 'redis' matchmaker from any zmq:// transport because it's fails in cases zmq+etcd:// or zmq+sentinel:// urls. 2) Allow to use transport_url from config file by removing default value for args.url option. In this case by default args.url is None and transport_url will be used from config file. Change-Id: I1edcabe46b1e92dd4e6eccc9151b50d03f394186
* | [simulator] Fix a message length generator usageKirill Bespalov2016-10-181-4/+2
|/ | | | | | | Do initialization of the generator only for rpc/notify clients. No needs to load messages_length.yaml and calculate distribution for servers. Change-Id: Ib1ac30181bec20eacf6d29ccd5f76c0b6b1e96f8
* [simulator] Automatic stopping of rpc-serversKirill Bespalov2016-10-061-19/+119
| | | | | | | | | | | | | | | | | | | This patch provide the sync flag: simulator.py rpc-client --sync call simulator.py rpc-client --sync fanout The --sync values means next: - call: to send sync msg via rpc.call - fanout: to broadcast sync msg via rpc.fanout to all servers on topic When clients has sent all messages, the rpc-server will be stopped automatically and dump statistics to a file (if --json-file is used). This is much usefull than rough killing the process of the server in benchmark frameworks. Change-Id: I06fd8dbbcdc8b2b9f13029029f730b417ff128ce
* Fix simulator bool command line argsGevorg Davoian2016-09-301-5/+4
| | | | | | | | | | | | | | --debug, --is-cast and --is-fanout are defined as args of type=bool. This means that, for example, if we want to enable debug logging level, we have to type '--debug True'. But we can also use '--debug False' in order to do the same, which is very misleading (in fact, any non-empty string will evaluate to True). This patch tries to solve this problem by replacing type=bool and default=False with action='store_true' for these args, so that we will be able to enable them (they will remain False by default as before) simply as '--debug' etc. Change-Id: I8ee04c35427df446966161491da8d264b44975bf
* Fix calculating of duration in simulator.pykbespalov2016-08-161-2/+11
| | | | | | | | | | | | | | | | | | | When we calculate metrics like msg/sec, latency, etc we expect the start and end time as time of the first and last processed message: RPC Server life timeline: [----0..5 sec----][---5..10 sec---][---10..15 sec--] waiting clients 10 msg recved wait sigint expected: duration 5 sec, 2 msg/sec actual (incorrect): duration 15 sec, 0.6 msg/sec no reason to set the boundaries if server was idle few seconds before running of clients and after. Change-Id: I33e0a605b54ea7b89977504892528c41c3b00a68
* Move zmq driver options into its own groupOleksii Zamiatin2016-08-051-1/+1
| | | | | | | | | | ZeroMQ driver options are current stored into the DEFAULT group. This change makes the zmq configuration clearer by putting its options into oslo_messaging_zmq group. Change-Id: Ia00fda005b1664750d2646f8c82ebdf295b156fb Closes-bug: #1417040 Co-Authored-By: Oleksii Zamiatin <ozamiatin@mirantis.com>
* Properly cleanup listener and driver on simulator exitGevorg Davoian2016-07-241-13/+19
| | | | | | Change-Id: Id04d4d1ce131bf7a4681273c438cbe6e58b44e78 Closes-Bug: #1584743 Co-Authored-By: Oleksii Zamiatin <ozamiatin@mirantis.com>
* Merge "Fix simulator stat printing"Jenkins2016-05-081-46/+56
|\
| * Fix simulator stat printingYulia Portnova2016-05-061-46/+56
| | | | | | | | Change-Id: I97f05f64475541ec9269c17644cb27b46ef4f1cf
* | Merge "Simulator: align stats to whole seconds"Jenkins2016-05-051-1/+9
|\ \ | |/ |/|
| * Simulator: align stats to whole secondsIlya Shakhat2016-03-221-1/+9
| | | | | | | | | | | | | | | | Aligning message sending and stats collection to whole seconds. This allows to merge data from client and server - useful for visualization of message flow. Change-Id: I507c021d851254d4d84e8922de687931f8545864
* | Fixes sumulator.py signal_handler logicDmitriy Ukhlov2016-04-291-44/+54
| | | | | | | | | | Change-Id: I95ee737a6c12572e7b3d27d8ea117c9f2909ef28 Closes-bug: #1576614
* | Merge "Simulator: handle SIGINT and SIGTERM signals"Jenkins2016-04-201-13/+34
|\ \
| * | Simulator: handle SIGINT and SIGTERM signalsIlya Shakhat2016-04-191-13/+34
| | | | | | | | | | | | | | | | | | | | | | | | Now it is possible to stop simulator client and server by sending SIGINT or SIGTERM signals. Note that both stop gracefully and it takes some time to do this. Change-Id: Ie3fd1ea8b146070d61a247fd8ccc124df8d34848
* | | Allow simulator to be launched from arbitrary directoryDmitry Mescheryakov2016-04-071-1/+2
|/ / | | | | | | | | | | | | | | | | | | | | Right now simulator.py expects messages_length.yaml to be in the current directory and as a result, it is impossible to launch simulator while you are at the repository root. The change makes simulator to search for the file in the same directory as simulator itself is in. Change-Id: Ic2ca21d87e2d888432dc6204c7b4b995a06aecbd
* | Merge "Simulator: collect error stats"Jenkins2016-03-281-1/+16
|\ \
| * | Simulator: collect error statsIlya Shakhat2016-03-231-1/+16
| | | | | | | | | | | | | | | | | | Collect client-side error statistics. Change-Id: I8fb41e3a5e58a9215961775be3c70d7c1a822285
* | | Simulator: make parameter wait_after_msg floatIlya Shakhat2016-03-231-2/+2
|/ / | | | | | | | | | | Introduce more precise control on message sending delay, Change-Id: I7bc2e575d5fe29c4940fef9b28e2e25df68b3095
* | Support python3 in simulator.pyDavanum Srinivas2016-03-211-6/+7
|/ | | | | | | | | | | | | | | | Tested with: python3 simulator.py --url \ rabbit://stackrabbit:flopsymopsy@localhost:5672/ notify-server python3 simulator.py --url \ rabbit://stackrabbit:flopsymopsy@localhost:5672/ notify-client -m 1000 and: python3 simulator.py --url \ rabbit://stackrabbit:flopsymopsy@localhost:5672/ rpc-server python3 simulator.py --url \ rabbit://stackrabbit:flopsymopsy@localhost:5672/ rpc-client -m 1000 Change-Id: I1f8bb964aef23867a651e192dc355635e36f78a1
* Simulator: store results in JSON formatIlya Shakhat2016-03-101-7/+22
| | | | | | | This patch introduces new parameter --json <file name>. When it is specified the simulator stores stats in JSON file. Change-Id: I0b7a87ee614f44baa920569fe0c7046269c90925
* Simulator: calculate message latency statisticsIlya Shakhat2016-03-101-160/+314
| | | | | | | | | | | | | | | | | | | | | Calculate message latency: for RPC calls on both server-side and on client-side (round-trip); for RPC cast and NOTIFY on server-side only. The message is extended with metadata: sequence id, creation time, reception time and return time (for RPC calls). For every message the metadata is stored into MessageStatCollector. To reduce memory consumption the collector aggregates stats on every second and can print the ongoing stats to stdout. At the finish the overall stats are printed. NOTIFY tests are changed to be similar to RPC: * use pre-generated messages * parameter "-w WAIT_BEFORE_ANSWER" for processing delay * parameter "--requeue" to re-queue new messages (only those that miss the cache) Change-Id: I4c2fc11bfaee17b6825cdc7d6edf07b8e91ef83a
* Simulator: always use random messages for time-bound testsIlya Shakhat2016-03-031-7/+11
| | | | | | | When the test is time-bound the simulator should use as many random messages as possible. Change-Id: I1e112d9ff0c3f1281b544f05d25eff2d3912fd6c
* Simulator: implement own random generator instead of scipyIlya Shakhat2016-03-021-15/+23
| | | | | | | | Dependency on scipy results in extra requirements to the system (include the need of Fortran compiler). With this patch the weighted random choice algorithm is implemented directly. Change-Id: I60cbc6c452945000add5a65263d1ab4e42dd91f9
* Simulator: fix batch-notify-server commandIlya Shakhat2016-03-021-1/+1
| | | | | | | This patch fixes the name of parameters used to construct batch notification listener Change-Id: I8eae9494943b2592c9cf4b92fccca93eab3b8e93
View Lorry Controller Status