HADOOP-3693. Fix archives, distcp and native library documentation to

conform to style guidelines. Contributed by Amareshwari Sriramadasu.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.18@674642 13f79535-47bb-0310-9956-ffa450edef68

CHANGES.txt

@@ -720,6 +720,9 @@ Release 0.18.0 - Unreleased
     input. Validation job still runs on default fs.
     (Jothi Padmanabhan via cdouglas)
 
+    HADOOP-3693. Fix archives, distcp and native library documentation to
+    conform to style guidelines. (Amareshwari Sriramadasu via cdouglas)
+
 Release 0.17.1 - Unreleased
 
   INCOMPATIBLE CHANGES

docs/changes.html

@@ -305,7 +305,7 @@ InputFormat.validateInput.<br />(tomwhite via omalley)</li>
     </ol>
   </li>
   <li><a href="javascript:toggleList('release_0.18.0_-_unreleased_._bug_fixes_')">  BUG FIXES
-</a>&nbsp;&nbsp;&nbsp;(119)
+</a>&nbsp;&nbsp;&nbsp;(120)
     <ol id="release_0.18.0_-_unreleased_._bug_fixes_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2905">HADOOP-2905</a>. 'fsck -move' triggers NPE in NameNode.<br />(Lohit Vjayarenu via rangadi)</li>
       <li>Increment ClientProtocol.versionID missed by <a href="http://issues.apache.org/jira/browse/HADOOP-2585">HADOOP-2585</a>.<br />(shv)</li>
@@ -543,6 +543,8 @@ a lock during this call.<br />(Arun C Murthy via cdouglas)</li>
 read from DFS.<br />(rangadi)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-3683">HADOOP-3683</a>. Fix dfs metrics to count file listings rather than files
 listed.<br />(lohit vijayarenu via cdouglas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-3597">HADOOP-3597</a>. Fix SortValidator to use filesystems other than the default as
+input. Validation job still runs on default fs.<br />(Jothi Padmanabhan via cdouglas)</li>
     </ol>
   </li>
 </ul>

docs/distcp.html

@@ -234,10 +234,10 @@ document.write("Last Published: " + document.lastModified);
 <h2 class="h3">Overview</h2>
 <div class="section">
 <p>DistCp (distributed copy) is a tool used for large inter/intra-cluster
-      copying. It uses map/reduce to effect its distribution, error
-      handling/recovery, and reporting. It expands a list of files and
+      copying. It uses Map/Reduce to effect its distribution, error
+      handling and recovery, and reporting. It expands a list of files and
       directories into input to map tasks, each of which will copy a partition
-      of the files specified in the source list. Its map/reduce pedigree has
+      of the files specified in the source list. Its Map/Reduce pedigree has
       endowed it with some quirks in both its semantics and execution. The
       purpose of this document is to offer guidance for common tasks and to
       elucidate its model.</p>
@@ -303,13 +303,13 @@ document.write("Last Published: " + document.lastModified);
         copier failed for some subset of its files, but succeeded on a later
         attempt (see <a href="#etc">Appendix</a>).</p>
 <p>It is important that each TaskTracker can reach and communicate with
-        both the source and destination filesystems. For hdfs, both the source
+        both the source and destination file systems. For HDFS, both the source
         and destination must be running the same version of the protocol or use
         a backwards-compatible protocol (see <a href="#cpver">Copying Between
         Versions</a>).</p>
 <p>After a copy, it is recommended that one generates and cross-checks
         a listing of the source and destination to verify that the copy was
-        truly successful. Since DistCp employs both map/reduce and the
+        truly successful. Since DistCp employs both Map/Reduce and the
         FileSystem API, issues in or between any of the three could adversely
         and silently affect the copy. Some have had success running with
         <span class="codefrag">-update</span> enabled to perform a second pass, but users should
@@ -518,7 +518,7 @@ document.write("Last Published: " + document.lastModified);
           copiers (i.e. maps) may not always increase the number of
           simultaneous copies nor the overall throughput.</p>
 <p>If <span class="codefrag">-m</span> is not specified, DistCp will attempt to
-          schedule work for <span class="codefrag">min(total_bytes / bytes.per.map, 20 *
+          schedule work for <span class="codefrag">min (total_bytes / bytes.per.map, 20 *
           num_task_trackers)</span> where <span class="codefrag">bytes.per.map</span> defaults
           to 256MB.</p>
 <p>Tuning the number of maps to the size of the source and

docs/hadoop_archives.html

@@ -207,7 +207,7 @@ document.write("Last Published: " + document.lastModified);
 <div class="section">
 <p>
         Hadoop archives are special format archives. A Hadoop archive
-        maps to a FileSystem directory. A Hadoop archive always has a *.har
+        maps to a file system directory. A Hadoop archive always has a *.har
         extension. A Hadoop archive directory contains metadata (in the form 
         of _index and _masterindex) and data (part-*) files. The _index file contains
         the name of the files that are part of the archive and the location
@@ -224,20 +224,21 @@ document.write("Last Published: " + document.lastModified);
         
 </p>
 <p>
-        -archiveName is the name of the archive you would like to create. An example would be 
-        foo.har. The name should have a *.har extension. The inputs are filesystem pathnames which 
-        work as usual with regular expressions. The destination directory would contain the archive.
-        Note that this is a Map Reduce job that creates the archives. You would need a map reduce cluster
-        to run this. The following is an example:</p>
+        -archiveName is the name of the archive you would like to create. 
+        An example would be foo.har. The name should have a *.har extension. 
+        The inputs are file system pathnames which work as usual with regular
+        expressions. The destination directory would contain the archive.
+        Note that this is a Map/Reduce job that creates the archives. You would
+        need a map reduce cluster to run this. The following is an example:</p>
 <p>
         
 <span class="codefrag">hadoop archive -archiveName foo.har /user/hadoop/dir1 /user/hadoop/dir2 /user/zoo/</span>
         
 </p>
 <p>
-        In the above example /user/hadoop/dir1 and /user/hadoop/dir2 will be archived in the following
-        filesystem directory -- /user/zoo/foo.har. The sources are not changed or removed when an archive
-        is created.
+        In the above example /user/hadoop/dir1 and /user/hadoop/dir2 will be
+        archived in the following file system directory -- /user/zoo/foo.har.
+        The sources are not changed or removed when an archive is created.
         </p>
 </div>
         
@@ -245,9 +246,10 @@ document.write("Last Published: " + document.lastModified);
 <h2 class="h3"> How to look up files in archives? </h2>
 <div class="section">
 <p>
-        The archive exposes itself as a filesystem layer. So all the fs shell commands in the archives work but 
-        with a different URI. Also, note that archives are immutable. So, rename's, deletes and creates return an error. 
-        URI for Hadoop Archives is 
+        The archive exposes itself as a file system layer. So all the fs shell
+        commands in the archives work but with a different URI. Also, note that
+        archives are immutable. So, rename's, deletes and creates return
+        an error. URI for Hadoop Archives is 
         </p>
 <p>
 <span class="codefrag">har://scheme-hostname:port/archivepath/fileinarchive</span>

docs/hadoop_archives.pdf

@@ -58,10 +58,10 @@ endobj
 >>
 endobj
 14 0 obj
-<< /Length 1800 /Filter [ /ASCII85Decode /FlateDecode ]
+<< /Length 1798 /Filter [ /ASCII85Decode /FlateDecode ]
  >>
 stream
-Gatm<?#SIU'Rf_Zd+a_VQ6khF_9a>GQ8JbFClO?]=GDBmBhPKPWh^?Lqt6^8ah3kZ36d2jKXA[Em`.?^;X[Ym`GK.r^?o%'mEDhY1Y6ZcT5-#!Yo!8.m<YJ6.]bie>l2s<US-GXpasIm>[^+_i]Z-p>/;;E1j^MG]1_HcC\?0CLXkaLYgQ_OERn:_Hohg*b<N]pO'4B!=R4YDZZe":^iE9W:^)VZgQ+C3-DLd(]HTq%F1JQPXAD(Fh?P[jS97\H'Y"4ACC2,Js5%?J\NfSP'RaLkn!r$"]m#s)Oeel@n4GZ"hr=e=P[^daFChA;aX)548]`bl^=)?S[a)Wq);%^Q?UDpSB-dOXO;`%V%Br!#A0hLq9%;qD(#@%4s2rInGleQ#%J[3D,5CO"aMXk5TeV?1Z[.'(+5_r4_E(i-cntaAXr[99Na]=caON*cMROlaLu)>n-nR"FRusqW9.r8D>#d7Ob+gskE%U?Emd.8@D=aZ^9,QDl*uF#M("NrXirMu_eAJrOFr]#&L-SYne%M?6#pZr\S=4\=(9RWhmDqYIE4JhV,=T//HK>Lhd.RZHrP\5d>RRU+e!%,9<POi>ml0D-cA/P6qV>%PIEPdap'Rl[K\(-<iA2m^00"6?qiR:1-J<l"b=NEhBEQD12E<1c4SqgD9,lA,O>l``M[D4SFR31(:gXcU3B4dM[+3#JXnC!1<'<Ib$;J6,?PPA"=Z*UdLE.8[[h$TK3c2-QGP5a:Ufr6OnY3kW4_:7:*UnNq)qZQC,0L]G]-^TbWYj]s:D6;^.B9>bX;pI,BGH8tor@e7=kQtKK>7s$7mBhkG#]926ik=aI?4:DK9l7oCmkKspGZBI%dtmu`#d19nbY,\-jeR?d1m35ld9E5iZfm;8`$h&:erq!P\P_BC#.3"Vu*RN)'V'%kOhE,$o7(*!Uh:(YE["taDte<E5ECHBa[d]Xu8I=,dh%X"r\TW&#G2*dd"fFAD<Z[N7Sb9a$e!*i3hBC#PgdG*7/soX[-pMKPo70H''JrV5XKd8nl@1T9$gt5s;![S2MoQcM/]>lq;?/K<A'Jmu.tQkdSRFd#g`LVkhgDoVAP(ddNR<3BOlu@[[S?\?k;1A9c3F"fgBXUTj%tH2^"i$/TsNpb^R?0>*Bl%[at2cZX`a-R`%-estf/BJA4<FQJbI"+`=M&$K3Af[]d7(uQiocR#qH"J#tke*=BEG0j,lCPQl-%<gQXg/M9+E-@F^[tKkV4T"C'*?9BG:>[H7DsV=R0#Umi1c8V&7#<-J>JLf]q@*[6)j6E=i4>;P_SL@`R3ld4Oh+s*=qSb&bcL:T2B5!@nQkd!iblc^[2UC$mLPtEls)1Z4@r+1jsfD7li)#q=hX#dg!3cZ[N<CA1R.Y*4TKb#)\!#>[P2]o(69g88l@D?Qu*OkL+28t.?"p0,HdOf58_<V"BfS:ap<V[0q>CWVB8hqAV!/s&?9ttU]-Ya3Ijq3JmP&X2tgd=^*h,9Em<A@RKsV/dfc]<<Q@sCYMoaPd@k9$\T+dDG#bdN<GhN40MRKFJ^PY5Ncp9:W9Z;D5ptkr^j<2L&Z'F<D[\qr"o2rTm.u6Js4cnmH)$cQ:N2C1gHJ$c8##/EfSWq[G0t@.\M=eMAeZGf=L'Hdhd\`d`&nqAI7&0o'X[7P41&LACblX2]\'RE"r-_WJ&+4@(dnA1lZ=:llki0?!'ULsipsGb"&$i</onS!^8KM4A\7S:cV`K;iW04c>Tq.EjJoE3DK/V6"Kq4X[bgb\W8"kZ6E6't6Q<\hp#g44[*?FULdXAJ#tr5)?c>Hi!<~>
+Gatm<D/\/e&H;*)Tl3S^=I,me#=F8g=I0FuZ*c3=@2T8QVCep5W01$un(3W-,jQk@G+&m9'%.E]T"B?@46l3J[WiTkY&?DQC--L<h9<8Hn`O(K8*krDB=hJ`XU=BT/(]5]bu#VXYC9,a4fg=^_r;/E.';Eg8X_dTrnT*j:M]XPm"a-XMm+BWrA\SpGeP2RU#dA'Qq,RD7]c:Lf@aBm?spODF\m2j\[ETf^6souI,l#617Sjo>KR9f;>WP-I!ZIbYYj<cR'%l_'\LWMkgG%6V!)9`icJWKFZ;;t,=tV_iJ!<NI#efR[1m\aDm/OBZXHFsC/XV]Q7k(AEa+;P=9c4_iCdiej/`AY'r8hg>Q`k)Y-JZC2^N2WLDAVTs8S@L)tF,:/2uM-!fQ!Cj!,)<$kJ>gW%b:8YMnLsTs/!,dPVKjfX9`Ln(95"-Do0ed$D0aHF.E`Ug*=I,BY<.B,in5.MsPpd@;Pn(3MG"m'Aa<q:!8.N:!j-MfOX=<ZgG`!DpJZ_n=WF*7UR75Z7%6?I-iK&R6iZ-A]nZ(8.KEH)R&`2i;FIB\64d@=]BO?8&OT38Tc:@5W-fEd"hTejJ6)NHr#)^+G"U[o?DcEgF@`%FuK8lSka2)77p1S3.,Xa@mD6lp$ftUl#Yh!G+Te*#)DEI4".*#'C75NBH0oJrCTg]@IVE(Be%;i4"qC/KE5C&!j-Yq'f?Xnl>OdR*!LNWD22BhadtFe>`'Q]Y.+I,h4(T'T@A!HceMSMr=Pg2m@R"8e3FAG"o,kWu0e.h*`5Yg+$hm.n<24RYDu6SN`Vu0O"oA#je2hqOU%1X846_$38EEh=>2`/O#7jPCR0BVJFYKD9871UDZ4D5JrGCOH%g?0n[9;[,,L@iM/\iP.Zp::sUuLP\OGsg"b"<Vu*RN)$2`Ckfl\[$p*X2=mr.V0@[t$ahu,miIif:dh\\3>]"kYBFogd)eaWO5EmcQBr`&*"3Ktje[q\T4!mRsEF.C[FY9;2RfaG0V_na?RBUKG.515r:2+J$]M*%l5+dmEaRELIA54)0(^L)(66<#6gd.#9<'DN14[q[=1irrElr-gICO=Lf+:23\fNaqV4>Brk@GKT-ShnFJPQKNU.Q%63T@1oL1f#gP>K3QI-[t20c#^ZC[Q6JMLdj\@iaL)'%d"#6A`\;JjnM)K[=!Y:0h;:eBEk>Y9&Oas/IFcu6R9'CY,`bR'Xup_0P[ds2kWdnmL!??ZL*r3;M&OGICP&r:sS=5;Z"HlGN$9+'h)']F\\/\K!U.M6>O?p?u&,M;B)9e!A!N1.r$qUNJu?r1D#L!7U]m4HJ!lNRE$,ngBlYo?YVktYhni>('8Ta=R[R^"du9Qf+S'W79R=f67KsXKrds&fCXcG^+UUIBqsP#r>$^ieDelO?;i=C+1$s$g+^L92hc8:BZ*&ujbJ\#[ql+3<Z@>[emM5]V^n2""rL';TQ#u"<Q<bgs$D;W5c?S,02I<AHUEYH&S;<ZAsNJb"A79[(tEf3j*\&^F+^.L]TN0K3+>\\X4W$(ephJ&-H;$1l&WE3>iKOA]nT;/M>+$fHIU*id^B!g6Vog0rYdQ(52b/p2G^bp1t1R.*G!-t&&%KX;=f6Q7BU:8BK?'\)2[<.$5$7=Tg-`H[4N:OUIIQ$RN3=NZ8$:p`pA_&#/ZZBH^8J5;.oG9]H`;""FPB`2M@)2UW<t]=(57dUR#=rM.saoDL9n-ZoYF-QQGHQMIe=[:uDg6j>bPklDh3sgf?Ccl]6-ln%2WMm_#;]WfB#,)nabqkB:jPc,#j)6p(scQAFA3dE.B)~>
 endstream
 endobj
 15 0 obj
@@ -194,33 +194,33 @@ endobj
 xref
 0 28
 0000000000 65535 f 
-0000005333 00000 n 
-0000005405 00000 n 
-0000005497 00000 n 
+0000005331 00000 n 
+0000005403 00000 n 
+0000005495 00000 n 
 0000000015 00000 n 
 0000000071 00000 n 
 0000000640 00000 n 
 0000000760 00000 n 
 0000000799 00000 n 
-0000005631 00000 n 
+0000005629 00000 n 
 0000000934 00000 n 
-0000005694 00000 n 
+0000005692 00000 n 
 0000001070 00000 n 
-0000005760 00000 n 
+0000005758 00000 n 
 0000001207 00000 n 
-0000003100 00000 n 
-0000003208 00000 n 
-0000003792 00000 n 
-0000005826 00000 n 
-0000003900 00000 n 
-0000004137 00000 n 
-0000004388 00000 n 
-0000004671 00000 n 
-0000004784 00000 n 
-0000004894 00000 n 
-0000005002 00000 n 
-0000005108 00000 n 
-0000005224 00000 n 
+0000003098 00000 n 
+0000003206 00000 n 
+0000003790 00000 n 
+0000005824 00000 n 
+0000003898 00000 n 
+0000004135 00000 n 
+0000004386 00000 n 
+0000004669 00000 n 
+0000004782 00000 n 
+0000004892 00000 n 
+0000005000 00000 n 
+0000005106 00000 n 
+0000005222 00000 n 
 trailer
 <<
 /Size 28
@@ -228,5 +228,5 @@ trailer
 /Info 4 0 R
 >>
 startxref
-5877
+5875
 %%EOF

docs/native_libraries.html

@@ -221,10 +221,10 @@ document.write("Last Published: " + document.lastModified);
 <h2 class="h3">Purpose</h2>
 <div class="section">
 <p>Hadoop has native implementations of certain components for reasons of 
-      both performace &amp; non-availability of Java implementations. These 
+      both performance and non-availability of Java implementations. These 
       components are available in a single, dynamically-linked, native library. 
       On the *nix platform it is <em>libhadoop.so</em>. This document describes 
-      the usage &amp; details on how to build the native libraries.</p>
+      the usage and details on how to build the native libraries.</p>
 </div>
     
     
@@ -273,7 +273,7 @@ document.write("Last Published: " + document.lastModified);
         </li>
         
 <li>
-          Ensure you have either or both of <strong>&gt;zlib-1.2</strong> and 
+          Make sure you have either or both of <strong>&gt;zlib-1.2</strong> and 
           <strong>&gt;lzo2.0</strong> packages for your platform installed; 
           depending on your needs.
         </li>

src/docs/src/documentation/content/xdocs/distcp.xml

@@ -29,10 +29,10 @@
       <title>Overview</title>
 
       <p>DistCp (distributed copy) is a tool used for large inter/intra-cluster
-      copying. It uses map/reduce to effect its distribution, error
-      handling/recovery, and reporting. It expands a list of files and
+      copying. It uses Map/Reduce to effect its distribution, error
+      handling and recovery, and reporting. It expands a list of files and
       directories into input to map tasks, each of which will copy a partition
-      of the files specified in the source list. Its map/reduce pedigree has
+      of the files specified in the source list. Its Map/Reduce pedigree has
       endowed it with some quirks in both its semantics and execution. The
       purpose of this document is to offer guidance for common tasks and to
       elucidate its model.</p>
@@ -85,14 +85,14 @@
         attempt (see <a href="#etc">Appendix</a>).</p>
 
         <p>It is important that each TaskTracker can reach and communicate with
-        both the source and destination filesystems. For hdfs, both the source
+        both the source and destination file systems. For HDFS, both the source
         and destination must be running the same version of the protocol or use
         a backwards-compatible protocol (see <a href="#cpver">Copying Between
         Versions</a>).</p>
 
         <p>After a copy, it is recommended that one generates and cross-checks
         a listing of the source and destination to verify that the copy was
-        truly successful. Since DistCp employs both map/reduce and the
+        truly successful. Since DistCp employs both Map/Reduce and the
         FileSystem API, issues in or between any of the three could adversely
         and silently affect the copy. Some have had success running with
         <code>-update</code> enabled to perform a second pass, but users should
@@ -253,7 +253,7 @@
           simultaneous copies nor the overall throughput.</p>
 
           <p>If <code>-m</code> is not specified, DistCp will attempt to
-          schedule work for <code>min(total_bytes / bytes.per.map, 20 *
+          schedule work for <code>min (total_bytes / bytes.per.map, 20 *
           num_task_trackers)</code> where <code>bytes.per.map</code> defaults
           to 256MB.</p>
 

src/docs/src/documentation/content/xdocs/hadoop_archives.xml

@@ -24,7 +24,7 @@
         <title> What are Hadoop archives? </title>
         <p>
         Hadoop archives are special format archives. A Hadoop archive
-        maps to a FileSystem directory. A Hadoop archive always has a *.har
+        maps to a file system directory. A Hadoop archive always has a *.har
         extension. A Hadoop archive directory contains metadata (in the form 
         of _index and _masterindex) and data (part-*) files. The _index file contains
         the name of the files that are part of the archive and the location
@@ -37,25 +37,27 @@
         <code>Usage: hadoop archive -archiveName name &lt;src&gt;* &lt;dest&gt;</code>
         </p>
         <p>
-        -archiveName is the name of the archive you would like to create. An example would be 
-        foo.har. The name should have a *.har extension. The inputs are filesystem pathnames which 
-        work as usual with regular expressions. The destination directory would contain the archive.
-        Note that this is a Map Reduce job that creates the archives. You would need a map reduce cluster
-        to run this. The following is an example:</p>
+        -archiveName is the name of the archive you would like to create. 
+        An example would be foo.har. The name should have a *.har extension. 
+        The inputs are file system pathnames which work as usual with regular
+        expressions. The destination directory would contain the archive.
+        Note that this is a Map/Reduce job that creates the archives. You would
+        need a map reduce cluster to run this. The following is an example:</p>
         <p>
         <code>hadoop archive -archiveName foo.har /user/hadoop/dir1 /user/hadoop/dir2 /user/zoo/</code>
         </p><p>
-        In the above example /user/hadoop/dir1 and /user/hadoop/dir2 will be archived in the following
-        filesystem directory -- /user/zoo/foo.har. The sources are not changed or removed when an archive
-        is created.
+        In the above example /user/hadoop/dir1 and /user/hadoop/dir2 will be
+        archived in the following file system directory -- /user/zoo/foo.har.
+        The sources are not changed or removed when an archive is created.
         </p>
         </section>
         <section>
         <title> How to look up files in archives? </title>
         <p>
-        The archive exposes itself as a filesystem layer. So all the fs shell commands in the archives work but 
-        with a different URI. Also, note that archives are immutable. So, rename's, deletes and creates return an error. 
-        URI for Hadoop Archives is 
+        The archive exposes itself as a file system layer. So all the fs shell
+        commands in the archives work but with a different URI. Also, note that
+        archives are immutable. So, rename's, deletes and creates return
+        an error. URI for Hadoop Archives is 
         </p><p><code>har://scheme-hostname:port/archivepath/fileinarchive</code></p><p>
         If no scheme is provided it assumes the underlying filesystem. 
         In that case the URI would look like 

src/docs/src/documentation/content/xdocs/native_libraries.xml

@@ -29,10 +29,10 @@
       <title>Purpose</title>
       
       <p>Hadoop has native implementations of certain components for reasons of 
-      both performace &amp; non-availability of Java implementations. These 
+      both performance and non-availability of Java implementations. These 
       components are available in a single, dynamically-linked, native library. 
       On the *nix platform it is <em>libhadoop.so</em>. This document describes 
-      the usage &amp; details on how to build the native libraries.</p>
+      the usage and details on how to build the native libraries.</p>
     </section>
     
     <section>
@@ -68,7 +68,7 @@
           <a href="#Building+Native+Hadoop+Libraries">build</a> them yourself.
         </li>
         <li>
-          Ensure you have either or both of <strong>&gt;zlib-1.2</strong> and 
+          Make sure you have either or both of <strong>&gt;zlib-1.2</strong> and 
           <strong>&gt;lzo2.0</strong> packages for your platform installed; 
           depending on your needs.
         </li>