| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
| |
Signed-off-by: Kezhu Wang <kezhuw@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
|
|
|
| |
Signed-off-by: tison <wander4096@gmail.com>
|
|
|
|
|
|
|
| |
FileSnap.deserialize is effectively a static method. Change it to a
static method so that we don't need to do some new FileSnap(null)
hacks for workaround calling this method.
Signed-off-by: tison <wander4096@gmail.com>
|
|
|
|
|
|
| |
(Thread.suspend has been removed) (#1985)
* checkstyle
|
|
|
| |
Co-authored-by: Colvin Cowie <COLVINCO@uk.ibm.com>
|
|
|
| |
Signed-off-by: tison <wander4096@gmail.com>
|
|
|
| |
Author: Li Wang <liwang@apple.com>
|
|
|
|
|
|
|
| |
The semantics of persistent recursive watch promise no child events on
descendant nodes. When there are standard child watches on descendants
of node being watches in persistent recursive mode, server will deliver
child events to client inevitably. So we have to filter out child events
for persistent recursive watches on client side.
|
|
|
|
|
|
|
| |
InetAddress (#1959)
- upgrade Mockito to 4.9.0
- use BurningWave DNS mock tools
|
|
|
| |
Co-authored-by: Moderne <team@moderne.io>
|
|
|
|
|
| |
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
Signed-off-by: Enrico Olivelli <eolivelli@apache.org>
|
|
|
|
|
|
|
| |
Provides a restore command for restoring database from a snapshot
Author: Li Wang <liwang@apple.com>
Co-authored-by: liwang <liwang@apple.com>
|
|
|
|
| |
some other files are for 4.1.86 (#1974)
|
|
|
|
|
|
| |
ClientCnxn$EventThread.processEvent"
This reverts commit 86690ff40cb5c9f5782f0971db16e8fd1c3528e6.
|
| |
|
|
|
| |
Co-authored-by: Mate Szalay-Beko <symat@apache.com>
|
|
|
|
|
|
|
|
| |
Provides a snapshot command for taking snapshot and streaming out data
Author: Li Wang <liwang@apple.com>
Co-authored-by: Li Wang <liwang@apple.com>
Co-authored-by: Enrico Olivelli <eolivelli@apache.org>
|
|
|
|
|
|
|
| |
Co-authored-by: Mike Dobozy <mdobozy@redhat.com>
Co-authored-by: Anup Ghatage <ghatage@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
Reviewed-by: Mate Szalay-Beko <symat@apache.org>
|
|
|
|
| |
Author: Li Wang <liwang@apple.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
https://issues.apache.org/jira/browse/ZOOKEEPER-4303
Allows specifying an explicit port 0 for the client port or secure client port, which will default to operating system behavior of finding an unbound port. Modified ZKServerEmbedded to report the actual port instead of the configured port.
Author: Mike Drob <mdrob@apple.com>
Reviewers: Kezhu Wang <kezhuw@gmail.com>, Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>
Closes #1868 from madrob/zookeeper-4303
|
|
|
|
|
|
|
|
|
|
| |
This is the first step mentioned in [ZOOKEEPER-102](https://issues.apache.org/jira/browse/ZOOKEEPER-102) and we should not play with ByteBuffer among the request handling code path.
Author: tison <wander4096@gmail.com>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>
Closes #1905 from tisonkun/request-supplier
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR tries to fix several test failures in `RequestThrottlerTest`.
First, `RequestThrottlerTest#testDropStaleRequests`.
Place `Thread.sleep(200)` after `submittedRequests.take()` in `RequestThrottler#run` will fail two assertions:
1. `assertEquals(2L, (long) metrics.get("prep_processor_request_queued"))`
2. `assertEquals(1L, (long) metrics.get("request_throttle_wait_count"))`
This happens due to `setStale` chould happen before throttle handling.
This commit solves this by introducing an interception point `RequestThrottler.throttleSleep` to build happen-before relations:
1. `throttling.countDown` happens before `setStale`, this ensures that unthrottled request are processed as usual.
2. `setStale` happens before `throttled.await`, this defends `RequestThrottler.throttleSleep` against spurious wakeup.
Second, `RequestThrottlerTest#testRequestThrottler`.
* `RequestThrottlerTest.testRequestThrottler:197 expected: <2> but was: <1>`
`ZooKeeperServer#submitRequest` and `PrepRequestProcessor#processRequest` run in different threads, thus there is no guarantee on metric `prep_processor_request_queued` after `submitted.await(5, TimeUnit.SECONDS)`. Place `Thread.sleep(200)` before `zks.submitRequestNow(request)` in `RequestThrottler#run` will incur this failure.
* `RequestThrottlerTest.testRequestThrottler:206 expected: <5> but was: <4>`
`entered.await(STALL_TIME, TimeUnit.MILLISECONDS)` could return `false` due to almost same timeout as `RequestThrottler#throttleSleep`. Place `Thread.sleep(500)` around `throttleSleep` will increase failure possibility.
Third, `RequestThrottlerTest#testGlobalOutstandingRequestThrottlingWithRequestThrottlerDisabled`.
* `RequestThrottlerTest.testGlobalOutstandingRequestThrottlingWithRequestThrottlerDisabled:340 expected: <3> but was: <4>`
`ZooKeeperServer#shouldThrottle` depends on consistent sum of `getInflight` and `getInProcess`. But it is no true. Place `Thread.sleep(200)` before `zks.submitRequestNow(request)` in `RequestThrottler#run` could reproduce this.
Sees also https://github.com/apache/zookeeper/pull/1739, https://github.com/apache/zookeeper/pull/1821.
Author: Kezhu Wang <kezhuw@gmail.com>
Reviewers: Mate Szalay-Beko <symat@apache.org>, maoling <maoling@apache.org>
Closes #1887 from kezhuw/ZOOKEEPER-4327-flaky-RequestThrottlerTest.testDropStaleRequests
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
```java
if (serverPath.length() > chrootPath.length()) {
event.setPath(serverPath.substring(chrootPath.length()));
}
```
Currently, chroot strip code listed above could result in illegal path
(aka. path not start with "/"). This will disconnect zookeeper client
due to `StringIndexOutOfBoundsException` from `PathParentIterator.next`
in event handling.
Author: Kezhu Wang <kezhuw@gmail.com>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>
Closes #1899 from kezhuw/ZOOKEEPER-4565-refine-chroot-strip
|
|
|
|
|
|
|
|
|
|
|
| |
add a tool to recursively collect and display child count and data
stored in sub-trees
Author: Szabolcs Bukros <szabolcs@cloudera.com>
Reviewers: Andor Molnar <andor@apache.org>, Mate Szalay-Beko <symat@apache.org>
Closes #1902 from BukrosSzabolcs/ZOOKEEPER-4566
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch is based on #1903.
This closes #1903.
Author: tison <wander4096@gmail.com>
Reviewers: Andor Molnar <andor@apache.org>, Mate Szalay-Beko <symat@apache.org>
Closes #1904 from tisonkun/encapsulate-request-bytebuffer
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
According to [this comment in ZOOKEEPER-102](https://issues.apache.org/jira/browse/ZOOKEEPER-102?focusedCommentId=16977000&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16977000) I introduce a `Protocol` abstraction and going to moving all wire protocol concept into `cnxn` and this scope, so that client and server's business logics handle only deserialized/real record.
cc eolivelli maoling Randgalt
This supersedes #1832.
Author: tison <wander4096@gmail.com>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>
Closes #1837 from tisonkun/protocol
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
FileTxnSnapLogMetricsTest.testFileTxnSnapLogMetrics
This test writes txns to trigger snapshot and expects some txns remain
in txn log. But snapshot taking is asynchronous, thus all txns could be
written to snapshot. So in restarting, it is possible that no txns to
load after snapshot restored. This will fail assertion.
This commit solves this by disable automic snapshot taking by
`SyncRequestProcessor.setSnapCount(Integer.MAX_VALUE)`.
Author: Kezhu Wang <kezhuw@gmail.com>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>
Closes #1852 from kezhuw/ZOOKEEPER-4511-FileTxnSnapLogMetricsTest-testFileTxnSnapLogMetrics
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are several problems in this test:
* It uses `ParameterizedTest` which run tests in single jvm. But
`ZooKeeperServer.enableEagerACLCheck` is `static` and loaded from env
variable.
* It uses `assertNotSame` which assert on object reference equiality.
* It asserts on `zkLeader.getLastLoggedZxid()` while client connect to
`connectedServer`. There is no happen-before between
`zkLeader.getLastLoggedZxid()` and successful response from other
server. The commit and response are routed to different servers and
performed asynchronous in each server.
Author: Kezhu Wang <kezhuw@gmail.com>
Reviewers: maoling <maoling199210191@sina.com>, Mate Szalay-Beko <symat@apache.org>
Closes #1851 from kezhuw/ZOOKEEPER-4308-EagerACLFilterTest
|
|
|
|
| |
This reverts commit 3fd25d4ebc854328ccc3f8d1c42f6e4c63a84fac.
|
|
|
|
|
|
|
|
|
|
| |
We run two ZooKeeperServerEmbedded in one JVM and find that ProviderRegistry was initialized repeatedly.
Author: lanicc <lanettiesso@gmail.com>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, tison <wander4096@gmail.com>, maoling <maoling@apache.org>
Closes #1888 from lanicc/ZOOKEEPER-4549
|
|
|
|
|
|
|
|
|
|
| |
ZOOKEEPER-4518 : remove useless log in the PrepRequestProcessor#pRequest method
Author: vinayvinayu <vinayvinayu@gmail.com>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, maoling <maoling@apache.org>
Closes #1862 from Vinayvinayu/ZOOKEEPER-4518
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Motivation:
See more here https://issues.apache.org/jira/browse/ZOOKEEPER-4551
Modifications:
Do not log the stacktrace, just write the message at INFO level
Author: Enrico Olivelli <eolivelli@apache.org>
Reviewers: Mate Szalay-Beko <symat@apache.org>
Closes #1889 from eolivelli/fix/remove-spam-stacktrace
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This test failed following assertions in ci:
1. `RequestThrottlerTest.testRequestThrottler:206 expected: <5> but was: <4>`
This is caused by no happens-before relationship between
`connectionLossCount` and `disconnected.await`. Places
`disconnected.countDown()` after `connectionLossCount++` to solve
this.
2. `RequestThrottlerTest.testLargeRequestThrottling:297 expected: <2> but was: <0>`
Large request throttling is handled in io thread, while
`prep_processor_request_queued` metric is updated in processor
thread. Places metric assertion after `finished.await` to solve this.
Additionally, I find one more potential flaky case. After connection
closed due to throttling third request, reconnecting could fail this
test in slow sending environment. It is easy to reproduce by adding
`Thread.sleep(i * 100)` in sending loop.
Author: Kezhu Wang <kezhuw@gmail.com>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>
Closes #1821 from kezhuw/ZOOKEEPER-4327-flaky-RequestThrottlerTest.testLargeRequestThrottling and squashes the following commits:
e21c2f8f7 [Kezhu Wang] ZOOKEEPER-4327: Fix flaky RequestThrottlerTest.testDropStaleRequests
3df34b827 [Kezhu Wang] ZOOKEEPER-4327: Fix flaky RequestThrottlerTest.testLargeRequestThrottling
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Zookeeper server can get stuck when it has just one client and the only way it recovers is due to a socket timeout or another client commit request.
Sync thread reads commitIsWaiting outside of a sync block and acts on this information in a sync block later. The actual status of commitIsWaiting can change between the time where commitIsWaiting is read and acted upon because commit thread updates it outside a sync block.
Fix here is to ensure that we read and process commitIsWaiting inside a sync block.
https://issues.apache.org/jira/browse/ZOOKEEPER-4537
Author: jithin23 <jithin.girish@gmail.com>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>
Closes #1877 from jithin23/ZOOKEEPER-4537
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is backwards compatible, since the new method on the `Testable` interface defaults to a no-op.
Author: Houston Putman <houston@apache.org>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, maoling <maoling@apache.org>
Closes #1863 from HoustonPutman/zk-testable-socket-close and squashes the following commits:
1f9d814f5 [Houston Putman] Fix style
475c9f516 [Houston Putman] ZOOKEEPER-4519: Add closeSocket method to Testable interface
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ZooKeer currently has support for reloading the Quorum Truststore & Keystore automatically when the certificate files change in the filesystem without server restart (https://github.com/apache/zookeeper/pull/680)
However, Reloading of key and trust store for **ClientX509Util** is not present; i.e., the server presented certs to the clients will not get reloaded automatically if the certificates in the filesystem change, short-lived certs requires the process restart.
Changes:
- A new config property "zookeeper.client.certReload" is added, if it's true - ClientX509Util is reloaded automatically.
- ZK uses an _X509AuthenticationProvider_ which is backed by an X509TrustManager and an X509KeyManager to perform _remote host certificate authentication_. We need to update the X509AuthenticationProvider's TrustStore as part of the X509Util file-watcher.
- Junit test case to verify the cert reload.
Author: Manu Mathew <manu.mathew@netapp.com>
Author: mathewmanu <manmathew@cs.stonybrook.edu>
Author: Manu Mathew <101424654+mathew-manu@users.noreply.github.com>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>
Closes #1839 from mathew-manu/ZOOKEEPER-3806
|
|
|
|
|
|
|
|
|
|
|
| |
Removed tcnative depdency as it is required after netty upgrade to 4.1.76
Removed tcnative CVE supressions
Author: Brahma Reddy Battula <brahma@apache.org>
Reviewers: Mohammad Arshad <arshad@apache.org>
Closes #1873 from brahmareddybattula/master
|
|
|
|
|
|
|
|
|
|
| |
upgrading netty in master as per review in : https://github.com/apache/zookeeper/pull/1867
Author: Ananya Singh <purpul90@gmail.com>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, Shoothzj <shoothzj@gmail.com>, Mohammad Arshad <arshad@apache.org>, Brahma Reddy Battula <brahma@apache.org>
Closes #1869 from AnanyaSingh2121/ZK-4529
|
|
|
|
|
|
|
|
|
|
|
| |
Author: Mohammad Arshad <arshad@apache.org>
Reviewers: Mate Szalay-Beko <symat@apache.org>, Enrico Olivelli <eolivelli@apache.org>
Closes #1855 from arshadmohammad/ZOOKEEPER-1875-npe and squashes the following commits:
4a7d471e3 [Mohammad Arshad] Corrected impacted test cases
12f44d62e [Mohammad Arshad] ZOOKEEPER-1875: NullPointerException in ClientCnxn$EventThread.processEvent
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1. For connection closing state scenario, changed the log level to debug
2. When JVM exiting with code 0, then logging info instead of error
Author: Mohammad Arshad <arshad@apache.org>
Reviewers: tison <wander4096@gmail.com>, Enrico Olivelli <eolivelli@apache.org>
Closes #1856 from arshadmohammad/ZOOKEEPER-4515-cli and squashes the following commits:
e7e248bd5 [Mohammad Arshad] Logging error only when exit code is non zero
31e124a2a [Mohammad Arshad] ZOOKEEPER-4515: ZK Cli quit command always logs error 1. For connection closing state scenario, changed the log level to debug 2. When JVM exiting with code 0, then logging info instead of error
|
|
|
|
|
|
|
|
|
|
| |
Moved channel object null check to sendPkt method to cover all calling scenarios
Author: Mohammad Arshad <arshad@apache.org>
Reviewers: Mate Szalay-Beko <symat@apache.org>
Closes #1854 from arshadmohammad/ZOOKEEPER-4514-npe
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Make ZKUtil#deleteRecursive API fully compatible with older versions
Author: Mohammad Arshad <arshad@apache.org>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>
Closes #1843 from arshadmohammad/ZOOKEEPER-4504-DeleteRecursive and squashes the following commits:
851bb1ee9 [Mohammad Arshad] Added javadoc for ZKUtil#deleteRecursive(zk, pathRoot, batchSize) API
e7b33116c [Mohammad Arshad] Added test case to verify ZKUtil.deleteRecursive() in sync and async mode
008b2bd4a [Mohammad Arshad] ZOOKEEPER-4504: ZKUtil#deleteRecursive causing deadlock in HDFS HA functionality
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Changes:
* Format `OpCode.addWatch` as "addWatch" in `Request.op2String`.
Author: Kezhu Wang <kezhuw@gmail.com>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, maoling <maoling@apache.org>
Closes #1819 from kezhuw/ZOOKEEPER-4467-op_code_addWatch_string and squashes the following commits:
97f891da1 [Kezhu Wang] fixup! ZOOKEEPER-4467: Format OpCode.addWatch in Request.op2String
ebe6faa30 [Kezhu Wang] ZOOKEEPER-4467: Format OpCode.addWatch in Request.op2String
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
internal value
When packets are added to ClientCnxn's outgoing packet queue we ensure there's no conflict with an ongoing flush of that queue because of connection loss.
Synchronization used to be on the state field's value. This value is both not stable (its value changes over time), possibly causing improper synchronization, and global, which can cause contention in applications that run several ZooKeeper clients.
We now synchronize on outgoingQueue which is both local to a ClientCnxn's instance and stable.
Author: Sylvain Wallez <sylvain@bluxte.net>
Reviewers: maoling <maoling@apache.org>, Mohammad Arshad <arshad@apache.org>
Closes #1257 from swallez/ZOOKEEPER-3652 and squashes the following commits:
82e2cad2c [Sylvain Wallez] Instruct SpotBugs that we know what we're doing when synchronizing on outgoingQueue
b0bc03d6f [Sylvain Wallez] ZOOKEEPER-3652: Synchronize ClientCnxn outgoing queue flush on a stable internal value
|
|
|
|
|
|
|
|
|
|
| |
cc maoling eolivelli
Author: tison <wander4096@gmail.com>
Reviewers: maoling <maoling@apache.org>
Closes #1838 from tisonkun/patch-2
|
|
|
|
|
|
|
|
|
|
|
| |
Changes:
* Unused `ZooDefs.opNames` dropped.
Author: Kezhu Wang <kezhuw@gmail.com>
Reviewers: maoling <maoling@apache.org>
Closes #1822 from kezhuw/ZOOKEEPER-4474-drop-unused-ZooDefs.opNames
|
|
|
|
|
|
|
|
|
|
|
| |
- rename commos-io LICENSE file to 2.11.0
- create file for commons-cli (ASF project)
Author: Enrico Olivelli <eolivelli@apache.org>
Reviewers: Mate Szalay-Beko <symat@apache.org>, maoling <maoling@apache.org>
Closes #1829 from eolivelli/ZOOKEEPER-4482
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
future renewals since Java 9
This bug is similar to the one fixed in https://issues.apache.org/jira/browse/KAFKA-12730.
Our Kerberos ticket refresh thread performs re-login by logging out and then logging in again. If
login fails, we retry after some sleep. Every reLogin() operation performs loginContext.logout()
and loginContext.login(). If login fails, we end up with two consecutive logouts. This used to
work in older Java versions, but from Java 9 onwards, this results in a NullPointerException due
to https://bugs.openjdk.java.net/browse/JDK-8173069. We should check if logout is required before
attempting logout.
I fixed the issue and added a new unit test to test some ticket renewal scenarios. I managed to
reproduce the problem in KerberosTicketRenewalTest.shouldRecoverIfKerberosNotAvailableForSomeTime()
which (before the fix) failed with Java13 but succeeded with Java8.
Author: Mate Szalay-Beko <symat@apache.org>
Reviewers: Enrico Olivelli <eolivelli@apache.org>
Closes #1828 from symat/ZOOKEEPER-4477-master
|
|
|
|
|
|
|
|
|
|
| |
I think that the id of this task should be `build-time`. And it has original be `build-time`.
Author: Shoothzj <shoothzj@gmail.com>
Reviewers: Enrico Olivelli <eolivelli@apache.org>, maoling <maoling@apache.org>
Closes #1781 from Shoothzj/fix-typo-in-zookeeper-server-pom
|