diff options
| author | Andy Smith <ansmith@redhat.com> | 2018-09-17 13:21:42 -0400 |
|---|---|---|
| committer | Andy Smith <ansmith@redhat.com> | 2018-12-04 11:25:07 -0500 |
| commit | 5a842ae15582e4eedfb1b2510eaf4a8997701f58 (patch) | |
| tree | 283024cba8b0b6b736f85978aba5568280af1d51 /releasenotes | |
| parent | 252844879d9e34c81b1777a92ad4407fab4a6853 (diff) | |
| download | oslo-messaging-5a842ae15582e4eedfb1b2510eaf4a8997701f58.tar.gz | |
Switch driver to confluent-kafka client library
This patch switches the kafka python client from kafka-python to
confluent-kafka due to documented threading issues with the
kafka-python consumer and the recommendation to use multiplrocessing.
The confluent-kafka client leverages the high performance librdkafka
C client and is safe for multiple thread use.
This patch:
* switches to confluent-kafka library
* revises consumer and producer message operations
* utilizes event.tpool method for confluent-kafka blocking calls
* updates unit tests
* adds kafka specific timeouts for functional tests
* adds release note
Depends-On: Ice374dca539b8ed1b1965b75379bad5140121483
Change-Id: Idfb9fe3700d882c8285c6dc56b0620951178eba2
Diffstat (limited to 'releasenotes')
| -rw-r--r-- | releasenotes/notes/kafka-client-library-change-fe16d5a34550db7f.yaml | 13 |
1 files changed, 13 insertions, 0 deletions
diff --git a/releasenotes/notes/kafka-client-library-change-fe16d5a34550db7f.yaml b/releasenotes/notes/kafka-client-library-change-fe16d5a34550db7f.yaml new file mode 100644 index 0000000..ed2fcae --- /dev/null +++ b/releasenotes/notes/kafka-client-library-change-fe16d5a34550db7f.yaml @@ -0,0 +1,13 @@ +--- +fixes: + - | + Threading issues with the kafka-python consumer client were identified + and documented. The driver has been updated to integrate the + confluent-kafka python library. The confluent-kafka client + leverages the high performance librdkafka C client and is safe + for multiple thread use. +upgrade: + - | + With the change in the client library used, projects using the + Kafka driver should use extras oslo.messaging[kafka] to pull in + dependencies for the driver. |