HADOOP-4438. Update forrest documentation to include missing FsShell

commands. Contributed by Suresh Srinivas.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@706463 13f79535-47bb-0310-9956-ffa450edef68

CHANGES.txt

@@ -517,6 +517,9 @@ Release 0.19.0 - Unreleased
     HADOOP-4424. Change menu layout for Hadoop documentation (Boris Shkolnik
     via cdouglas).
 
+    HADOOP-4438. Update forrest documentation to include missing FsShell
+    commands. (Suresh Srinivas via cdouglas)
+
   OPTIMIZATIONS
 
     HADOOP-3556. Removed lock contention in MD5Hash by changing the 

docs/changes.html

@@ -113,7 +113,7 @@ classes.<br />(Suresh Srinivas via cdouglas)</li>
 </a></h2>
 <ul id="release_0.19.0_-_unreleased_">
   <li><a href="javascript:toggleList('release_0.19.0_-_unreleased_._incompatible_changes_')">  INCOMPATIBLE CHANGES
-</a>&nbsp;&nbsp;&nbsp;(20)
+</a>&nbsp;&nbsp;&nbsp;(21)
     <ol id="release_0.19.0_-_unreleased_._incompatible_changes_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-3595">HADOOP-3595</a>. Remove deprecated methods for mapred.combine.once
 functionality, which was necessary to providing backwards
@@ -172,6 +172,15 @@ WritableJobConf. Configuration.write is renamed to writeXml.<br />(omalley)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4281">HADOOP-4281</a>. Change dfsadmin to report available disk space in a format
 consistent with the web interface as defined in <a href="http://issues.apache.org/jira/browse/HADOOP-2816">HADOOP-2816</a>.<br />(Suresh
 Srinivas via cdouglas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-4430">HADOOP-4430</a>. Further change the cluster summary at name node web that was
+changed in <a href="http://issues.apache.org/jira/browse/HADOOP-2816">HADOOP-2816</a>:
+  Non DFS Used - This indicates the disk space taken by non DFS file from
+                 the Configured capacity
+  DFS Used % - DFS Used % of Configured Capacity
+  DFS Remaining % - Remaing % Configured Capacity available for DFS use
+DFS command line report reflects the same change. Config parameter
+dfs.datanode.du.pct is no longer used and is removed from the
+hadoop-default.xml.<br />(Suresh Srinivas via hairong)</li>
     </ol>
   </li>
   <li><a href="javascript:toggleList('release_0.19.0_-_unreleased_._new_features_')">  NEW FEATURES
@@ -262,7 +271,7 @@ changes from the prior release.<br />(cutting)</li>
     </ol>
   </li>
   <li><a href="javascript:toggleList('release_0.19.0_-_unreleased_._improvements_')">  IMPROVEMENTS
-</a>&nbsp;&nbsp;&nbsp;(74)
+</a>&nbsp;&nbsp;&nbsp;(77)
     <ol id="release_0.19.0_-_unreleased_._improvements_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4205">HADOOP-4205</a>. hive: metastore and ql to use the refactored SerDe library.<br />(zshao)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4106">HADOOP-4106</a>. libhdfs: add time, permission and user attribute support
@@ -395,6 +404,13 @@ incrementing the task attempt numbers by 1000 when the job restarts.<br />(Amar
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-3790">HADOOP-3790</a>. Add more unit tests for testing HDFS file append.<br />(szetszwo)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4150">HADOOP-4150</a>. Include librecordio in hadoop releases.<br />(Giridharan Kesavan
 via acmurthy)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-4321">HADOOP-4321</a>. Include documentation for the capacity scheduler.<br />(Hemanth
+Yamijala via omalley)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-4424">HADOOP-4424</a>. Change menu layout for Hadoop documentation (Boris Shkolnik
+via cdouglas).
+</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-4438">HADOOP-4438</a>. Update forrest documentation to include missing FsShell
+commands.<br />(Suresh Srinivas via cdouglas)</li>
     </ol>
   </li>
   <li><a href="javascript:toggleList('release_0.19.0_-_unreleased_._optimizations_')">  OPTIMIZATIONS
@@ -427,7 +443,7 @@ org.apache.hadoop.mapred  package private instead of public.<br />(omalley)</li>
     </ol>
   </li>
   <li><a href="javascript:toggleList('release_0.19.0_-_unreleased_._bug_fixes_')">  BUG FIXES
-</a>&nbsp;&nbsp;&nbsp;(139)
+</a>&nbsp;&nbsp;&nbsp;(140)
     <ol id="release_0.19.0_-_unreleased_._bug_fixes_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-3563">HADOOP-3563</a>.  Refactor the distributed upgrade code so that it is
 easier to identify datanode and namenode related code.<br />(dhruba)</li>
@@ -687,9 +703,13 @@ Fix the case when primary is dead.<br />(dhruba via szetszwo)</li>
 append.<br />(szetszwo)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4449">HADOOP-4449</a>. Fix dfsadmin usage.<br />(Raghu Angadi via cdouglas)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4455">HADOOP-4455</a>. Added TestSerDe so that unit tests can run successfully.<br />(Ashish Thusoo via dhruba)</li>
-      <li><a href="http://issues.apache.org/jira/browse/HADOOP-4399">HADOOP-4399</a>. Make fuse-dfs multi-thread access safe.<br />(Pete Wyckoff via dhruba)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4457">HADOOP-4457</a>. Fixes an input split logging problem introduced by
 <a href="http://issues.apache.org/jira/browse/HADOOP-3245">HADOOP-3245</a>.<br />(Amareshwari Sriramadasu via ddas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-4464">HADOOP-4464</a>. Separate out TestFileCreationClient from TestFileCreation.
+(Tsz Wo (Nicholas), SZE via cdouglas)
+</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-4404">HADOOP-4404</a>. saveFSImage() removes files from a storage directory that do
+not correspond to its type.<br />(shv)</li>
     </ol>
   </li>
 </ul>
@@ -699,7 +719,7 @@ append.<br />(szetszwo)</li>
 </a></h3>
 <ul id="release_0.18.2_-_unreleased_">
   <li><a href="javascript:toggleList('release_0.18.2_-_unreleased_._bug_fixes_')">  BUG FIXES
-</a>&nbsp;&nbsp;&nbsp;(8)
+</a>&nbsp;&nbsp;&nbsp;(9)
     <ol id="release_0.18.2_-_unreleased_._bug_fixes_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4116">HADOOP-4116</a>. Balancer should provide better resource management.<br />(hairong)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-3614">HADOOP-3614</a>. Fix a bug that Datanode may use an old GenerationStamp to get
@@ -714,6 +734,7 @@ ArrayIndexOutOfBoundsException.<br />(hairong)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4292">HADOOP-4292</a>. Do not support append() for LocalFileSystem.<br />(hairong)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4398">HADOOP-4398</a>. No need to truncate access time in INode. Also fixes NPE
 in CreateEditsLog.<br />(Raghu Angadi)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-4399">HADOOP-4399</a>. Make fuse-dfs multi-thread access safe.<br />(Pete Wyckoff via dhruba)</li>
     </ol>
   </li>
 </ul>

docs/hdfs_shell.html

@@ -224,6 +224,9 @@ document.write("Last Published: " + document.lastModified);
 <a href="#copyToLocal"> copyToLocal</a>
 </li>
 <li>
+<a href="#count"> count </a>
+</li>
+<li>
 <a href="#cp"> cp </a>
 </li>
 <li>
@@ -251,7 +254,10 @@ document.write("Last Published: " + document.lastModified);
 <a href="#mkdir"> mkdir </a>
 </li>
 <li>
-<a href="#movefromLocal"> movefromLocal </a>
+<a href="#moveFromLocal"> moveFromLocal </a>
+</li>
+<li>
+<a href="#moveToLocal"> moveToLocal</a>
 </li>
 <li>
 <a href="#mv"> mv </a>
@@ -376,7 +382,7 @@ document.write("Last Published: " + document.lastModified);
 <span class="codefrag">Usage: hadoop fs -copyFromLocal &lt;localsrc&gt; URI</span>
 			
 </p>
-<p>Similar to <a href="#putlink"><strong>put</strong></a> command, except that the source is restricted to a local file reference. </p>
+<p>Similar to <a href="#put"><strong>put</strong></a> command, except that the source is restricted to a local file reference. </p>
 <a name="N100BA"></a><a name="copyToLocal"></a>
 <h3 class="h4"> copyToLocal</h3>
 <p>
@@ -384,8 +390,45 @@ document.write("Last Published: " + document.lastModified);
 <span class="codefrag">Usage: hadoop fs -copyToLocal [-ignorecrc] [-crc] URI &lt;localdst&gt;</span>
 			
 </p>
-<p> Similar to <a href="#getlink"><strong>get</strong></a> command, except that the destination is restricted to a local file reference.</p>
-<a name="N100CF"></a><a name="cp"></a>
+<p> Similar to <a href="#get"><strong>get</strong></a> command, except that the destination is restricted to a local file reference.</p>
+<a name="N100CF"></a><a name="count"></a>
+<h3 class="h4"> count </h3>
+<p>
+				
+<span class="codefrag">Usage: hadoop fs -count [-q]  &lt;paths&gt;</span>
+			
+</p>
+<p>
+				Count the number of directories, files and bytes under the paths that match the specified file pattern. The output columns are:<br>
+<span class="codefrag">DIR_COUNT, FILE_COUNT, CONTENT_SIZE FILE_NAME</span>. <br>
+<br>The output columns with <span class="codefrag">-q</span> are:<br>
+<span class="codefrag">QUOTA, REMAINING_QUATA, SPACE_QUOTA, REMAINING_SPACE_QUOTA, DIR_COUNT, FILE_COUNT, CONTENT_SIZE, FILE_NAME</span>.
+		   </p>
+<p>Example:</p>
+<ul>
+				
+<li>
+					
+<span class="codefrag"> hadoop fs -count hdfs://nn1.example.com/file1 hdfs://nn2.example.com/file2 
+		   </span>
+				
+</li>
+				
+<li>
+					
+<span class="codefrag"> hadoop fs -count -q hdfs://nn1.example.com/file1
+		   </span>
+				
+</li>
+			
+</ul>
+<p>Exit Code:</p>
+<p>
+				
+<span class="codefrag"> Returns 0 on success and -1 on error.</span>
+			
+</p>
+<a name="N10108"></a><a name="cp"></a>
 <h3 class="h4"> cp </h3>
 <p>
 				
@@ -417,7 +460,7 @@ document.write("Last Published: " + document.lastModified);
 <span class="codefrag"> Returns 0 on success and -1 on error.</span>
 			
 </p>
-<a name="N100F9"></a><a name="du"></a>
+<a name="N10132"></a><a name="du"></a>
 <h3 class="h4">du</h3>
 <p>
 				
@@ -433,7 +476,7 @@ document.write("Last Published: " + document.lastModified);
 <span class="codefrag"> Returns 0 on success and -1 on error. </span>
 <br>
 </p>
-<a name="N10114"></a><a name="dus"></a>
+<a name="N1014D"></a><a name="dus"></a>
 <h3 class="h4"> dus </h3>
 <p>
 				
@@ -443,7 +486,7 @@ document.write("Last Published: " + document.lastModified);
 <p>
 	    Displays a summary of file lengths.
 	   </p>
-<a name="N10124"></a><a name="expunge"></a>
+<a name="N1015D"></a><a name="expunge"></a>
 <h3 class="h4"> expunge </h3>
 <p>
 				
@@ -452,7 +495,7 @@ document.write("Last Published: " + document.lastModified);
 </p>
 <p>Empty the Trash. Refer to <a href="hdfs_design.html">HDFS Architecture</a> for more information on Trash feature.
 	   </p>
-<a name="N10138"></a><a name="get"></a>
+<a name="N10171"></a><a name="get"></a>
 <h3 class="h4"> get </h3>
 <p>
 				
@@ -488,7 +531,7 @@ document.write("Last Published: " + document.lastModified);
 <span class="codefrag"> Returns 0 on success and -1 on error. </span>
 			
 </p>
-<a name="N1016C"></a><a name="getmerge"></a>
+<a name="N101A4"></a><a name="getmerge"></a>
 <h3 class="h4"> getmerge </h3>
 <p>
 				
@@ -498,7 +541,7 @@ document.write("Last Published: " + document.lastModified);
 <p>
 	  Takes a source directory and a destination file as input and concatenates files in src into the destination local file. Optionally <span class="codefrag">addnl</span> can be set to enable adding a newline character at the end of each file.  
 	  </p>
-<a name="N1017F"></a><a name="ls"></a>
+<a name="N101B7"></a><a name="ls"></a>
 <h3 class="h4"> ls </h3>
 <p>
 				
@@ -520,14 +563,14 @@ document.write("Last Published: " + document.lastModified);
 <span class="codefrag"> Returns 0 on success and -1 on error. </span>
 <br>
 </p>
-<a name="N101A2"></a><a name="lsr"></a>
+<a name="N101DA"></a><a name="lsr"></a>
 <h3 class="h4">lsr</h3>
 <p>
 <span class="codefrag">Usage: hadoop fs -lsr &lt;args&gt;</span>
 <br>
 	      Recursive version of <span class="codefrag">ls</span>. Similar to Unix <span class="codefrag">ls -R</span>.
 	      </p>
-<a name="N101B5"></a><a name="mkdir"></a>
+<a name="N101ED"></a><a name="mkdir"></a>
 <h3 class="h4"> mkdir </h3>
 <p>
 				
@@ -562,16 +605,23 @@ document.write("Last Published: " + document.lastModified);
 <span class="codefrag">Returns 0 on success and -1 on error.</span>
 			
 </p>
-<a name="N101E2"></a><a name="movefromLocal"></a>
-<h3 class="h4"> movefromLocal </h3>
+<a name="N1021A"></a><a name="moveFromLocal"></a>
+<h3 class="h4"> moveFromLocal </h3>
 <p>
 				
-<span class="codefrag">Usage: dfs -moveFromLocal &lt;src&gt; &lt;dst&gt;</span>
+<span class="codefrag">Usage: dfs -moveFromLocal &lt;localsrc&gt; &lt;dst&gt;</span>
 			
 </p>
-<p>Displays a "not implemented" message.
-	   </p>
-<a name="N101F2"></a><a name="mv"></a>
+<p>Similar to <a href="#put"><strong>put</strong></a> command, except that the source <span class="codefrag">localsrc</span> is deleted after it's copied. </p>
+<a name="N10232"></a><a name="moveToLocal"></a>
+<h3 class="h4"> moveToLocal</h3>
+<p>
+				
+<span class="codefrag">Usage: hadoop fs -moveToLocal [-crc] &lt;src&gt; &lt;dst&gt;</span>
+			
+</p>
+<p>Displays a "Not implemented yet" message.</p>
+<a name="N10242"></a><a name="mv"></a>
 <h3 class="h4"> mv </h3>
 <p>
 				
@@ -604,7 +654,7 @@ document.write("Last Published: " + document.lastModified);
 <span class="codefrag"> Returns 0 on success and -1 on error.</span>
 			
 </p>
-<a name="N1021C"></a><a name="put"></a>
+<a name="N1026C"></a><a name="put"></a>
 <h3 class="h4"> put </h3>
 <p>
 				
@@ -645,7 +695,7 @@ document.write("Last Published: " + document.lastModified);
 <span class="codefrag"> Returns 0 on success and -1 on error. </span>
 			
 </p>
-<a name="N10253"></a><a name="rm"></a>
+<a name="N102A2"></a><a name="rm"></a>
 <h3 class="h4"> rm </h3>
 <p>
 				
@@ -671,7 +721,7 @@ document.write("Last Published: " + document.lastModified);
 <span class="codefrag"> Returns 0 on success and -1 on error.</span>
 			
 </p>
-<a name="N10277"></a><a name="rmr"></a>
+<a name="N102C6"></a><a name="rmr"></a>
 <h3 class="h4"> rmr </h3>
 <p>
 				
@@ -702,7 +752,7 @@ document.write("Last Published: " + document.lastModified);
 <span class="codefrag"> Returns 0 on success and -1 on error. </span>
 			
 </p>
-<a name="N102A1"></a><a name="setrep"></a>
+<a name="N102F0"></a><a name="setrep"></a>
 <h3 class="h4"> setrep </h3>
 <p>
 				
@@ -728,7 +778,7 @@ document.write("Last Published: " + document.lastModified);
 <span class="codefrag">Returns 0 on success and -1 on error. </span>
 			
 </p>
-<a name="N102C6"></a><a name="stat"></a>
+<a name="N10315"></a><a name="stat"></a>
 <h3 class="h4"> stat </h3>
 <p>
 				
@@ -752,7 +802,7 @@ document.write("Last Published: " + document.lastModified);
 	   
 <span class="codefrag"> Returns 0 on success and -1 on error.</span>
 </p>
-<a name="N102E9"></a><a name="tail"></a>
+<a name="N10338"></a><a name="tail"></a>
 <h3 class="h4"> tail </h3>
 <p>
 				
@@ -776,7 +826,7 @@ document.write("Last Published: " + document.lastModified);
 	   
 <span class="codefrag"> Returns 0 on success and -1 on error.</span>
 </p>
-<a name="N1030C"></a><a name="test"></a>
+<a name="N1035B"></a><a name="test"></a>
 <h3 class="h4"> test </h3>
 <p>
 				
@@ -799,7 +849,7 @@ document.write("Last Published: " + document.lastModified);
 </li>
 			
 </ul>
-<a name="N1032F"></a><a name="text"></a>
+<a name="N1037E"></a><a name="text"></a>
 <h3 class="h4"> text </h3>
 <p>
 				
@@ -811,7 +861,7 @@ document.write("Last Published: " + document.lastModified);
 <p>
 	   Takes a source file and outputs the file in text format. The allowed formats are zip and TextRecordInputStream.
 	  </p>
-<a name="N10341"></a><a name="touchz"></a>
+<a name="N10390"></a><a name="touchz"></a>
 <h3 class="h4"> touchz </h3>
 <p>
 				

src/docs/src/documentation/content/xdocs/hdfs_shell.xml

@@ -92,14 +92,38 @@
 			<p>
 				<code>Usage: hadoop fs -copyFromLocal &lt;localsrc&gt; URI</code>
 			</p>
-			<p>Similar to <a href="#putlink"><strong>put</strong></a> command, except that the source is restricted to a local file reference. </p>
+			<p>Similar to <a href="#put"><strong>put</strong></a> command, except that the source is restricted to a local file reference. </p>
 		</section>
 		<section>
 			<title> copyToLocal</title>
 			<p>
 				<code>Usage: hadoop fs -copyToLocal [-ignorecrc] [-crc] URI &lt;localdst&gt;</code>
 			</p>
-			<p> Similar to <a href="#getlink"><strong>get</strong></a> command, except that the destination is restricted to a local file reference.</p>
+			<p> Similar to <a href="#get"><strong>get</strong></a> command, except that the destination is restricted to a local file reference.</p>
+		</section>
+		<section>
+			<title> count </title>
+			<p>
+				<code>Usage: hadoop fs -count [-q]  &lt;paths&gt;</code>
+			</p>
+			<p>
+				Count the number of directories, files and bytes under the paths that match the specified file pattern. The output columns are:<br/><code>DIR_COUNT, FILE_COUNT, CONTENT_SIZE FILE_NAME</code>. <br/><br/>The output columns with <code>-q</code> are:<br/><code>QUOTA, REMAINING_QUATA, SPACE_QUOTA, REMAINING_SPACE_QUOTA, DIR_COUNT, FILE_COUNT, CONTENT_SIZE, FILE_NAME</code>.
+		   </p>
+			<p>Example:</p>
+			<ul>
+				<li>
+					<code> hadoop fs -count hdfs://nn1.example.com/file1 hdfs://nn2.example.com/file2 
+		   </code>
+				</li>
+				<li>
+					<code> hadoop fs -count -q hdfs://nn1.example.com/file1
+		   </code>
+				</li>
+			</ul>
+			<p>Exit Code:</p>
+			<p>
+				<code> Returns 0 on success and -1 on error.</code>
+			</p>
 		</section>
 		<section>
 			<title> cp </title>
@@ -151,7 +175,7 @@
 	   </p>
 		</section>
 		<section>
-			<title id="getlink"> get </title>
+			<title> get </title>
 			<p>
 				<code>Usage: hadoop fs -get [-ignorecrc] [-crc] &lt;src&gt; &lt;localdst&gt;</code>
 				<br/>
@@ -227,12 +251,18 @@
 			</p>
 		</section>
 		<section>
-			<title> movefromLocal </title>
+			<title> moveFromLocal </title>
 			<p>
-				<code>Usage: dfs -moveFromLocal &lt;src&gt; &lt;dst&gt;</code>
+				<code>Usage: dfs -moveFromLocal &lt;localsrc&gt; &lt;dst&gt;</code>
 			</p>
-			<p>Displays a "not implemented" message.
-	   </p>
+			<p>Similar to <a href="#put"><strong>put</strong></a> command, except that the source <code>localsrc</code> is deleted after it's copied. </p>
+		</section>
+		<section>
+			<title> moveToLocal</title>
+			<p>
+				<code>Usage: hadoop fs -moveToLocal [-crc] &lt;src&gt; &lt;dst&gt;</code>
+			</p>
+			<p>Displays a "Not implemented yet" message.</p>
 		</section>
 		<section>
 			<title> mv </title>
@@ -258,7 +288,7 @@
 			</p>
 		</section>
 		<section>
-			<title id="putlink"> put </title>
+			<title> put </title>
 			<p>
 				<code>Usage: hadoop fs -put &lt;localsrc&gt; ... &lt;dst&gt;</code>
 			</p>