Преглед на файлове

ZOOKEEPER-3593: fix the default value of jute.maxbuffer in client side and an optimization for the documentation

- The default value of `jute.maxbuffer` in client side said it's 4MB, but actually, it was never working, because when the client reads the deserialized znode data, it also calls `checkLength(BinaryInputArchive.java:127)` where  `jute.maxbuffer` default value is 1MB. It's easy to reproduce, just read a znode more than 1MB with any special configure client. Look at the stack trace I attached in the JIRA
- Users also confused about that the doc said `jute.maxbuffer` must be set on all servers and clients, but their default value is not same in the [ZOOKEEPER-1295](https://issues.apache.org/jira/browse/ZOOKEEPER-1295)
- more details in the [ZOOKEEPER-3593](https://issues.apache.org/jira/browse/ZOOKEEPER-3593)

Author: maoling <maoling199210191@sina.com>

Reviewers: enixon@apache.org, eolivelli@apache.org

Closes #1129 from maoling/ZOOKEEPER-3593
maoling преди 5 години
родител
ревизия
ef3649f599

+ 16 - 8
zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md

@@ -1384,15 +1384,23 @@ the variable does.
     set to no, ZooKeeper will not require updates to be synced to
     the media.
 
-* *jute.maxbuffer:* :
-    (Java system property:**jute.maxbuffer**)
-    This option can only be set as a Java system property.
+* *jute.maxbuffer* :
+    (Java system property:**jute.maxbuffer**).
+    - This option can only be set as a Java system property.
     There is no zookeeper prefix on it. It specifies the maximum
-    size of the data that can be stored in a znode. The default is
-    0xfffff, or just under 1M. If this option is changed, the system
-    property must be set on all servers and clients otherwise
-    problems will arise. This is really a sanity check. ZooKeeper is
-    designed to store data on the order of kilobytes in size.
+    size of the data that can be stored in a znode. The unit is: byte. The default is
+    0xfffff(1048575) bytes, or just under 1M.
+    - If this option is changed, the system property must be set on all servers and clients otherwise
+    problems will arise.
+      - When *jute.maxbuffer* in the client side is greater than the server side, the client wants to write the data
+        exceeds *jute.maxbuffer* in the server side, the server side will get **java.io.IOException: Len error**
+      - When *jute.maxbuffer* in the client side is less than the server side, the client wants to read the data
+        exceeds *jute.maxbuffer* in the client side, the client side will get **java.io.IOException: Unreasonable length**
+        or **Packet len  is out of range!**
+    - This is really a sanity check. ZooKeeper is designed to store data on the order of kilobytes in size.
+      In the production environment, increasing this property to exceed the default value is not recommended for the following reasons:
+      - Large size znodes cause unwarranted latency spikes, worsen the throughput
+      - Large size znodes make the synchronization time between leader and followers unpredictable and non-convergent(sometimes timeout), cause the quorum unstable
 
 * *jute.maxbuffer.extrasize*:
     (Java system property: **zookeeper.jute.maxbuffer.extrasize**)

+ 4 - 3
zookeeper-docs/src/main/resources/markdown/zookeeperProgrammers.md

@@ -1305,10 +1305,11 @@ following reference
     and the password to unlock the file.
 
 * *jute.maxbuffer* :
-    It specifies the maximum size of the incoming data from the server. The default value is 4194304
-    Bytes , or just 4 MB. This is really a sanity check. The ZooKeeper server is designed to store and send
+    In the client side, it specifies the maximum size of the incoming data from the server. The default is 0xfffff(1048575) bytes,
+    or just under 1M. This is really a sanity check. The ZooKeeper server is designed to store and send
     data on the order of kilobytes. If incoming data length is more than this value, an IOException
-    is raised.
+    is raised. This value of client side should keep same with the server side(Setting **System.setProperty("jute.maxbuffer", "xxxx")** in the client side will work),
+    otherwise problems will arise.
 
 * *zookeeper.kinit* :
     Specifies path to kinit binary. Default is "/usr/bin/kinit".

+ 1 - 1
zookeeper-server/src/main/java/org/apache/zookeeper/client/ZKClientConfig.java

@@ -57,7 +57,7 @@ public class ZKClientConfig extends ZKConfig {
      */
     @SuppressWarnings("deprecation")
     public static final String SECURE_CLIENT = ZooKeeper.SECURE_CLIENT;
-    public static final int CLIENT_MAX_PACKET_LENGTH_DEFAULT = 4096 * 1024; /* 4 MB */
+    public static final int CLIENT_MAX_PACKET_LENGTH_DEFAULT = 0xfffff; /* 1 MB */
     public static final String ZOOKEEPER_REQUEST_TIMEOUT = "zookeeper.request.timeout";
     public static final String ZOOKEEPER_SERVER_PRINCIPAL = "zookeeper.server.principal";
     /**