ZOOKEEPER-437. Variety of Documentation Updates (grant via mahadev)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/zookeeper/trunk@783162 13f79535-47bb-0310-9956-ffa450edef68

CHANGES.txt

@@ -204,6 +204,8 @@ via mahadev)
  
   ZOOKEEPER-434. the java shell should indicate connection status on command prompt (henry robinson via breed)
 
+  ZOOKEEPER-437. Variety of Documentation Updates (grant via mahadev)
+
 NEW FEATURES:
 
   ZOOKEEPER-371. jdiff documentation included in build/release (giri via phunt)

docs/zookeeperAdmin.html

@@ -266,6 +266,9 @@ document.write("Last Published: " + document.lastModified);
 <a href="#sc_clusterOptions">Cluster Options</a>
 </li>
 <li>
+<a href="#sc_authOptions">Authentication &amp; Authorization Options</a>
+</li>
+<li>
 <a href="#Unsafe+Options">Unsafe Options</a>
 </li>
 </ul>
@@ -1018,9 +1021,22 @@ server.3=zoo3:2888:3888</span>
               and the configuration variable name is different from the system
               property. Yes - it's not consistent, and it's annoying.)</p>
 </dd>
+          
+<dt>
+<term>maxClientCnxns</term>
+</dt>
+<dd>
+<p>(No Java system property)</p>
+<p>Limits the number of concurrent connections (at the socket 
+              level) that a single client, identified by IP address, may make
+              to a single member of the ZooKeeper ensemble. This is used to 
+              prevent certain classes of DoS attacks, including file 
+              descriptor exhaustion. Setting this to 0 or omitting it entirely 
+              removes the limit on concurrent connections.</p>
+</dd>
         
 </dl>
-<a name="N1030B"></a><a name="sc_clusterOptions"></a>
+<a name="N10314"></a><a name="sc_clusterOptions"></a>
 <h4>Cluster Options</h4>
 <p>The options in this section are designed for use with an ensemble
         of servers -- that is, when deploying clusters of servers.</p>
@@ -1121,7 +1137,7 @@ server.3=zoo3:2888:3888</span>
               must be the ZooKeeper ensemble. </p>
 </dd>
 
-	
+          
 <dt>
 <term>weight.x=nnnnn</term>
 </dt>
@@ -1138,7 +1154,41 @@ server.3=zoo3:2888:3888</span>
         
 </dl>
 <p></p>
-<a name="N1037A"></a><a name="Unsafe+Options"></a>
+<a name="N10383"></a><a name="sc_authOptions"></a>
+<h4>Authentication &amp; Authorization Options</h4>
+<p>The options in this section allow control over
+        authentication/authorization performed by the service.</p>
+<dl>
+          
+<dt>
+<term>zookeeper.DigestAuthenticationProvider.superDigest</term>
+</dt>
+<dd>
+<p>(Java system property only: <strong>zookeeper.DigestAuthenticationProvider.superDigest</strong>)</p>
+<p>By default this feature is <strong>disabled</strong>
+</p>
+<p>
+<strong>New in 3.2:</strong>
+              Enables a ZooKeeper ensemble administrator to access the
+              znode hierarchy as a "super" user. In particular no ACL
+              checking occurs for a user authenticated as
+              super.</p>
+<p>org.apache.zookeeper.server.auth.DigestAuthenticationProvider
+              can be used to generate the superDigest, call it with
+              one parameter of "super:&lt;password&gt;". Provide the
+              generated "super:&lt;data&gt;" as the system property value
+              when starting each server of the ensemble.</p>
+<p>When authenticating to a ZooKeeper server (from a
+              ZooKeeper client) pass a scheme of "digest" and authdata
+              of "super:&lt;password&gt;". Note that digest auth passes
+              the authdata in plaintext to the server, it would be
+              prudent to use this authentication method only on
+              localhost (not over the network) or over an encrypted
+              connection.</p>
+</dd>
+        
+</dl>
+<a name="N103A6"></a><a name="Unsafe+Options"></a>
 <h4>Unsafe Options</h4>
 <p>The following options can be useful, but be careful when you use
         them. The risk of each is explained along with the explanation of what
@@ -1183,7 +1233,7 @@ server.3=zoo3:2888:3888</span>
 </dd>
         
 </dl>
-<a name="N103AC"></a><a name="sc_zkCommands"></a>
+<a name="N103D8"></a><a name="sc_zkCommands"></a>
 <h3 class="h4">ZooKeeper Commands: The Four Letter Words</h3>
 <p>ZooKeeper responds to a small set of commands. Each command is
       composed of four letters. You issue the commands to ZooKeeper via telnet
@@ -1247,7 +1297,7 @@ server.3=zoo3:2888:3888</span>
 <pre class="code">$ echo ruok | nc 127.0.0.1 5111
 imok
 </pre>
-<a name="N103EC"></a><a name="sc_dataFileManagement"></a>
+<a name="N10418"></a><a name="sc_dataFileManagement"></a>
 <h3 class="h4">Data File Management</h3>
 <p>ZooKeeper stores its data in a data directory and its transaction
       log in a transaction log directory. By default these two directories are
@@ -1255,7 +1305,7 @@ imok
       transaction log files in a separate directory than the data files.
       Throughput increases and latency decreases when transaction logs reside
       on a dedicated log devices.</p>
-<a name="N103F5"></a><a name="The+Data+Directory"></a>
+<a name="N10421"></a><a name="The+Data+Directory"></a>
 <h4>The Data Directory</h4>
 <p>This directory has two files in it:</p>
 <ul>
@@ -1301,14 +1351,14 @@ imok
         idempotent nature of its updates. By replaying the transaction log
         against fuzzy snapshots ZooKeeper gets the state of the system at the
         end of the log.</p>
-<a name="N10431"></a><a name="The+Log+Directory"></a>
+<a name="N1045D"></a><a name="The+Log+Directory"></a>
 <h4>The Log Directory</h4>
 <p>The Log Directory contains the ZooKeeper transaction logs.
         Before any update takes place, ZooKeeper ensures that the transaction
         that represents the update is written to non-volatile storage. A new
         log file is started each time a snapshot is begun. The log file's
         suffix is the first zxid written to that log.</p>
-<a name="N1043B"></a><a name="sc_filemanagement"></a>
+<a name="N10467"></a><a name="sc_filemanagement"></a>
 <h4>File Management</h4>
 <p>The format of snapshot and log files does not change between
         standalone ZooKeeper servers and different configurations of
@@ -1328,7 +1378,7 @@ imok
         this document for more details on setting a retention policy
         and maintenance of ZooKeeper storage.
         </p>
-<a name="N10450"></a><a name="sc_commonProblems"></a>
+<a name="N1047C"></a><a name="sc_commonProblems"></a>
 <h3 class="h4">Things to Avoid</h3>
 <p>Here are some common problems you can avoid by configuring
       ZooKeeper correctly:</p>
@@ -1382,7 +1432,7 @@ imok
 </dd>
       
 </dl>
-<a name="N10474"></a><a name="sc_bestPractices"></a>
+<a name="N104A0"></a><a name="sc_bestPractices"></a>
 <h3 class="h4">Best Practices</h3>
 <p>For best results, take note of the following list of good
       Zookeeper practices. <em>[tbd...]</em>

docs/zookeeperProgrammers.html

@@ -359,7 +359,7 @@ document.write("Last Published: " + document.lastModified);
 </li>
     
 </ul>
-<p>The next four sections of this provided practical programming
+<p>The next four sections provide practical programming
     information. These are:</p>
 <ul>
       
@@ -407,7 +407,7 @@ document.write("Last Published: " + document.lastModified);
     ZooKeeper application, you should probably at least read the chaptes on
     the <a href="#ch_zkDataModel">ZooKeeper Data Model</a> and <a href="#ch_guideToZkOperations">ZooKeeper Basic Operations</a>. Also,
     the <a href="#ch_programStructureWithExample">Simple Programmming
-    Example</a> <em>[tbd]</em> is helpful for understand the basic
+    Example</a> <em>[tbd]</em> is helpful for understanding the basic
     structure of a ZooKeeper client application.</p>
 </div>
 
@@ -469,7 +469,7 @@ document.write("Last Published: " + document.lastModified);
 </ul>
 <a name="N100A7"></a><a name="sc_zkDataModel_znodes"></a>
 <h3 class="h4">ZNodes</h3>
-<p>Every node in a ZooKeeper tree is refered to as a
+<p>Every node in a ZooKeeper tree is referred to as a
       <em>znode</em>. Znodes maintain a stat structure that
       includes version numbers for data changes, acl changes. The stat
       structure also has timestamps. The version number, together with the

docs/zookeeperStarted.html

@@ -337,7 +337,7 @@ clientPort=2181
 <strong>Java</strong>: Use</p>
 
           
-<pre class="code">bin/zkCli.sh 127.0.0.1:2181</pre>
+<pre class="code">bin/zkCli.sh -server 127.0.0.1:2181</pre>
 
           
 <p>This lets you perform simple, file-like operations.</p>
@@ -375,7 +375,130 @@ clientPort=2181
 </li>
       
 </ul>
-<a name="N100D4"></a><a name="sc_ProgrammingToZooKeeper"></a>
+<p>Once you have connected, you should see something like:
+        </p>
+<pre class="code">
+
+Connecting to localhost:2181
+log4j:WARN No appenders could be found for logger (org.apache.zookeeper.ZooKeeper).
+log4j:WARN Please initialize the log4j system properly.
+Welcome to ZooKeeper!
+JLine support is enabled
+[zkshell: 0]
+        </pre>
+<p>
+        From the shell, type <span class="codefrag command">help</span> to get a listing of commands that can be executed from the client, as in:
+      </p>
+<pre class="code">
+
+[zkshell: 0] help
+ZooKeeper host:port cmd args
+        get path [watch]
+        ls path [watch]
+        set path data [version]
+        delquota [-n|-b] path
+        quit
+        printwatches on|off
+        createpath data acl
+        stat path [watch]
+        listquota path
+        history
+        setAcl path acl
+        getAcl path
+        sync path
+        redo cmdno
+        addauth scheme auth
+        delete path [version]
+        setquota -n|-b val path
+
+        </pre>
+<p>From here, you can try a few simple commands to get a feel for this simple command line interface.  First, start by issuing the list command, as
+      in <span class="codefrag command">ls</span>, yielding:
+      </p>
+<pre class="code">
+
+[zkshell: 8] ls /
+[zookeeper]
+        </pre>
+<p>Next, create a new znode by running <span class="codefrag command">create /zk_test my_data</span>. This creates a new znode and associates the string "my_data" with the node.
+      You should see:</p>
+<pre class="code">
+
+[zkshell: 9] create /zk_test my_data
+Created /zk_test
+      </pre>
+<p>  Issue another <span class="codefrag command">ls /</span> command to see what the directory looks like:
+        </p>
+<pre class="code">
+
+[zkshell: 11] ls /
+[zookeeper, zk_test]
+
+        </pre>
+<p>
+      Notice that the zk_test directory has now been created.
+      </p>
+<p>Next, verify that the data was associated with the znode by running the <span class="codefrag command">get</span> command, as in:
+      </p>
+<pre class="code">
+
+[zkshell: 12] get /zk_test
+my_data
+cZxid = 5
+ctime = Fri Jun 05 13:57:06 PDT 2009
+mZxid = 5
+mtime = Fri Jun 05 13:57:06 PDT 2009
+pZxid = 5
+cversion = 0
+dataVersion = 0
+aclVersion = 0
+ephemeralOwner = 0
+dataLength = 7
+numChildren = 0
+        </pre>
+<p>We can change the data associated with zk_test by issuing the <span class="codefrag command">set</span> command, as in:
+        </p>
+<pre class="code">
+
+[zkshell: 14] set /zk_test junk
+cZxid = 5
+ctime = Fri Jun 05 13:57:06 PDT 2009
+mZxid = 6
+mtime = Fri Jun 05 14:01:52 PDT 2009
+pZxid = 5
+cversion = 0
+dataVersion = 1
+aclVersion = 0
+ephemeralOwner = 0
+dataLength = 4
+numChildren = 0
+[zkshell: 15] get /zk_test
+junk
+cZxid = 5
+ctime = Fri Jun 05 13:57:06 PDT 2009
+mZxid = 6
+mtime = Fri Jun 05 14:01:52 PDT 2009
+pZxid = 5
+cversion = 0
+dataVersion = 1
+aclVersion = 0
+ephemeralOwner = 0
+dataLength = 4
+numChildren = 0
+      </pre>
+<p>
+       (Notice we did a <span class="codefrag command">get</span> after setting the data and it did, indeed, change.</p>
+<p>Finally, let's <span class="codefrag command">delete</span> the node by issuing:
+      </p>
+<pre class="code">
+
+[zkshell: 16] delete /zk_test
+[zkshell: 17] ls /
+[zookeeper]
+[zkshell: 18]
+</pre>
+<p>That's it for now.  To explore more, continue with the rest of this document and see the <a href="zookeeperProgrammers.html">Programmer's Guide</a>. </p>
+<a name="N10130"></a><a name="sc_ProgrammingToZooKeeper"></a>
 <h3 class="h4">Programming to ZooKeeper</h3>
 <p>ZooKeeper has a Java bindings and C bindings. They are
       functionally equivalent. The C bindings exist in two variants: single
@@ -383,7 +506,7 @@ clientPort=2181
       is done. For more information, see the <a href="zookeeperProgrammers.html#ch_programStructureWithExample.html">Programming
       Examples in the ZooKeeper Programmer's Guide</a> for
       sample code using of the different APIs.</p>
-<a name="N100E2"></a><a name="sc_RunningReplicatedZooKeeper"></a>
+<a name="N1013E"></a><a name="sc_RunningReplicatedZooKeeper"></a>
 <h3 class="h4">Running Replicated ZooKeeper</h3>
 <p>Running ZooKeeper in standalone mode is convenient for evaluation,
       some development, and testing. But in production, you should run
@@ -443,7 +566,7 @@ server.3=zoo3:2888:3888
       
 </div>
 </div>
-<a name="N1011F"></a><a name="Other+Optimizations"></a>
+<a name="N1017B"></a><a name="Other+Optimizations"></a>
 <h3 class="h4">Other Optimizations</h3>
 <p>There are a couple of other configuration parameters that can
       greatly increase performance:</p>

src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml

@@ -78,7 +78,7 @@
       </listitem>
     </itemizedlist>
 
-    <para>The next four sections of this provided practical programming
+    <para>The next four sections provide practical programming
     information. These are:</para>
 
     <itemizedlist>
@@ -110,7 +110,7 @@
     the <ulink url="#ch_zkDataModel">ZooKeeper Data Model</ulink> and <ulink
     url="#ch_guideToZkOperations">ZooKeeper Basic Operations</ulink>. Also,
     the <ulink url="#ch_programStructureWithExample">Simple Programmming
-    Example</ulink> <emphasis>[tbd]</emphasis> is helpful for understand the basic
+    Example</ulink> <emphasis>[tbd]</emphasis> is helpful for understanding the basic
     structure of a ZooKeeper client application.</para>
   </section>
 
@@ -158,7 +158,7 @@
     <section id="sc_zkDataModel_znodes">
       <title>ZNodes</title>
 
-      <para>Every node in a ZooKeeper tree is refered to as a
+      <para>Every node in a ZooKeeper tree is referred to as a
       <emphasis>znode</emphasis>. Znodes maintain a stat structure that
       includes version numbers for data changes, acl changes. The stat
       structure also has timestamps. The version number, together with the

src/docs/src/documentation/content/xdocs/zookeeperStarted.xml

@@ -197,6 +197,128 @@ clientPort=2181
           system like operations on ZooKeeper.</para>
         </listitem>
       </itemizedlist>
+      <para>Once you have connected, you should see something like:
+        </para>
+      <programlisting>
+<![CDATA[
+Connecting to localhost:2181
+log4j:WARN No appenders could be found for logger (org.apache.zookeeper.ZooKeeper).
+log4j:WARN Please initialize the log4j system properly.
+Welcome to ZooKeeper!
+JLine support is enabled
+[zkshell: 0]
+]]>        </programlisting>
+      <para>
+        From the shell, type <command>help</command> to get a listing of commands that can be executed from the client, as in:
+      </para>
+      <programlisting>
+<![CDATA[
+[zkshell: 0] help
+ZooKeeper host:port cmd args
+        get path [watch]
+        ls path [watch]
+        set path data [version]
+        delquota [-n|-b] path
+        quit
+        printwatches on|off
+        createpath data acl
+        stat path [watch]
+        listquota path
+        history
+        setAcl path acl
+        getAcl path
+        sync path
+        redo cmdno
+        addauth scheme auth
+        delete path [version]
+        setquota -n|-b val path
+
+]]>        </programlisting>
+      <para>From here, you can try a few simple commands to get a feel for this simple command line interface.  First, start by issuing the list command, as
+      in <command>ls</command>, yielding:
+      </para>
+      <programlisting>
+<![CDATA[
+[zkshell: 8] ls /
+[zookeeper]
+]]>        </programlisting>
+      <para>Next, create a new znode by running <command>create /zk_test my_data</command>. This creates a new znode and associates the string "my_data" with the node.
+      You should see:</para>
+      <programlisting>
+<![CDATA[
+[zkshell: 9] create /zk_test my_data
+Created /zk_test
+]]>      </programlisting>
+      <para>  Issue another <command>ls /</command> command to see what the directory looks like:
+        </para>
+      <programlisting>
+<![CDATA[
+[zkshell: 11] ls /
+[zookeeper, zk_test]
+
+]]>        </programlisting><para>
+      Notice that the zk_test directory has now been created.
+      </para>
+      <para>Next, verify that the data was associated with the znode by running the <command>get</command> command, as in:
+      </para>
+      <programlisting>
+<![CDATA[
+[zkshell: 12] get /zk_test
+my_data
+cZxid = 5
+ctime = Fri Jun 05 13:57:06 PDT 2009
+mZxid = 5
+mtime = Fri Jun 05 13:57:06 PDT 2009
+pZxid = 5
+cversion = 0
+dataVersion = 0
+aclVersion = 0
+ephemeralOwner = 0
+dataLength = 7
+numChildren = 0
+]]>        </programlisting>
+      <para>We can change the data associated with zk_test by issuing the <command>set</command> command, as in:
+        </para>
+      <programlisting>
+<![CDATA[
+[zkshell: 14] set /zk_test junk
+cZxid = 5
+ctime = Fri Jun 05 13:57:06 PDT 2009
+mZxid = 6
+mtime = Fri Jun 05 14:01:52 PDT 2009
+pZxid = 5
+cversion = 0
+dataVersion = 1
+aclVersion = 0
+ephemeralOwner = 0
+dataLength = 4
+numChildren = 0
+[zkshell: 15] get /zk_test
+junk
+cZxid = 5
+ctime = Fri Jun 05 13:57:06 PDT 2009
+mZxid = 6
+mtime = Fri Jun 05 14:01:52 PDT 2009
+pZxid = 5
+cversion = 0
+dataVersion = 1
+aclVersion = 0
+ephemeralOwner = 0
+dataLength = 4
+numChildren = 0
+]]>      </programlisting>
+      <para>
+       (Notice we did a <command>get</command> after setting the data and it did, indeed, change.</para>
+      <para>Finally, let's <command>delete</command> the node by issuing:
+      </para>
+      <programlisting>
+<![CDATA[
+[zkshell: 16] delete /zk_test
+[zkshell: 17] ls /
+[zookeeper]
+[zkshell: 18]
+]]></programlisting>
+      <para>That's it for now.  To explore more, continue with the rest of this document and see the <ulink url="zookeeperProgrammers.html">Programmer's Guide</ulink>. </para>
     </section>
 
     <section id="sc_ProgrammingToZooKeeper">

src/docs/src/documentation/content/xdocs/zookeeperTutorial.xml

@@ -673,3 +673,5 @@ public class SyncPrimitive implements Watcher {
 </section>
 
 </article>
+
+        

src/java/main/org/apache/zookeeper/ZooKeeperMain.java

@@ -61,31 +61,31 @@ public class ZooKeeperMain {
     }
 
     static void populateCommandMap() {
-        commandMap.put("connect", " host:port");
+        commandMap.put("connect", "host:port");
         commandMap.put("close","");
         commandMap.put("create", "path data acl");
-        commandMap.put("delete"," path [version]");
-        commandMap.put("set"," path data [version]");
-        commandMap.put("get"," path [watch]");
-        commandMap.put("ls"," path [watch]");
-        commandMap.put("getAcl"," path");
-        commandMap.put("setAcl"," path acl");
-        commandMap.put("stat"," path [watch]");
-        commandMap.put("sync"," path");
-        commandMap.put("setquota"," -n|-b val path");
-        commandMap.put("listquota"," path");
-        commandMap.put("delquota"," [-n|-b] path");
+        commandMap.put("delete","path [version]");
+        commandMap.put("set","path data [version]");
+        commandMap.put("get","path [watch]");
+        commandMap.put("ls","path [watch]");
+        commandMap.put("getAcl","path");
+        commandMap.put("setAcl","path acl");
+        commandMap.put("stat","path [watch]");
+        commandMap.put("sync","path");
+        commandMap.put("setquota","-n|-b val path");
+        commandMap.put("listquota","path");
+        commandMap.put("delquota","[-n|-b] path");
         commandMap.put("history","");
-        commandMap.put("redo"," cmdno");
-        commandMap.put("printwatches", " on|off");
+        commandMap.put("redo","cmdno");
+        commandMap.put("printwatches", "on|off");
         commandMap.put("quit","");
-        commandMap.put("addauth", " scheme auth");
+        commandMap.put("addauth", "scheme auth");
     }
 
     static void usage() {
         System.err.println("ZooKeeper -server host:port cmd args");
         for (String cmd : commandMap.keySet()) {
-            System.err.println("\t"+cmd+commandMap.get(cmd));
+            System.err.println("\t"+cmd+ " " + commandMap.get(cmd));
         }
     }