HADOOP-9373. Merge change r1453599 from trunk

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1485850 13f79535-47bb-0310-9956-ffa450edef68

hadoop-common-project/hadoop-common/CHANGES.branch-trunk-win.txt

@@ -1,111 +0,0 @@
-branch-trunk-win changes - unreleased
-
-  HADOOP-8924. Hadoop Common creating package-info.java must not depend on sh.
-  (Chris Nauroth via suresh)
-
-  HADOOP-8945. Merge winutils from branch-1-win to branch-trunk-win.
-  (Bikas Saha, Chuan Liu, Giridharan Kesavan, Ivan Mitic, and Steve Maine
-  ported by Chris Nauroth via suresh)
-
-  HADOOP-8946. winutils: compile codebase during Maven build on 
-  branch-trunk-win. (Chris Nauroth via suresh)
-
-  HADOOP-8947. Merge FileUtil and Shell changes from branch-1-win to 
-  branch-trunk-win to enable initial test pass. (Raja Aluri, Davio Lao, 
-  Sumadhur Reddy Bolli, Ahmed El Baz, Kanna Karanam, Chuan Liu, 
-  Ivan Mitic, Chris Nauroth, and Bikas Saha via suresh)
-
-  HADOOP-8954. "stat" executable not found on Windows. (Bikas Saha, Ivan Mitic
-  ported by Chris Narouth via suresh)
-
-  HADOOP-8959. TestUserGroupInformation fails on Windows due to "id" executable
-  not found. (Bikas Saha, Ivan Mitic, ported by Chris Narouth via suresh)
-
-  HADOOP-8955. "chmod" executable not found on Windows.
-  (Chris Nauroth via suresh)
-
-  HADOOP-8960. TestMetricsServlet fails on Windows. (Ivan Mitic via suresh)
-
-  HADOOP-8961. GenericOptionsParser URI parsing failure on Windows.
-  (Ivan Mitic via suresh)
-
-  HADOOP-8949. Remove FileUtil.CygPathCommand dead code. (Chris Nauroth via 
-  suresh)
-
-  HADOOP-8956. FileSystem.primitiveMkdir failures on Windows cause multiple
-  test suites to fail. (Chris Nauroth via suresh)
-
-  HADOOP-8978. TestTrash fails on Windows. (Chris Nauroth via suresh)
-
-  HADOOP-8979. TestHttpServer fails on Windows. (Chris Nauroth via suresh)
-
-  HADOOP-8953. Shell PathData parsing failures on Windows. (Arpit Agarwal via
-  suresh)
-
-  HADOOP-8975. TestFileContextResolveAfs fails on Windows. (Chris Nauroth via
-  suresh)
-
-  HADOOP-8977. Multiple FsShell test failures on Windows. (Chris Nauroth via
-  suresh)
-
-  HADOOP-9005. Merge hadoop cmd line scripts from branch-1-win. (David Lao, 
-  Bikas Saha, Lauren Yang, Chuan Liu, Thejas M Nair and Ivan Mitic via suresh)
-
-  HADOOP-9008. Building hadoop tarball fails on Windows. (Chris Nauroth via
-  suresh)
-
-  HADOOP-9011. saveVersion.py does not include branch in version annotation.
-  (Chris Nauroth via suresh)
-
-  HADOOP-9110. winutils ls off-by-one error indexing MONTHS array can cause 
-  access violation. (Chris Nauroth via suresh)
-
-  HADOOP-9056. Build native library on Windows. (Chuan Liu, Arpit Agarwal via
-  suresh)
-
-  HADOOP-9144. Fix findbugs warnings. (Chris Nauroth via suresh)
-
-  HADOOP-9081. Add TestWinUtils. (Chuan Liu, Ivan Mitic, Chris Nauroth, 
-  and Bikas Saha via suresh)
-
-  HADOOP-9146. Fix sticky bit regression on branch-trunk-win.
-  (Chris Nauroth via suresh)
-
-  HADOOP-9266. Fix javac, findbugs, and release audit warnings on
-  branch-trunk-win. (Chris Nauroth via suresh)
-
-  HADOOP-9270. Remove a stale java comment from FileUtil. (Chris Nauroth via
-  szetszwo)
-
-  HADOOP-9271. Revert Python build scripts from branch-trunk-win.
-  (Chris Nauroth via suresh)
-
-  HADOOP-9313. Remove spurious mkdir from hadoop-config.cmd.
-  (Ivan Mitic via suresh)
-
-  HADOOP-9309. Test failures on Windows due to UnsatisfiedLinkError
-  in NativeCodeLoader#buildSupportsSnappy. (Arpit Agarwal via suresh)
-
-  HADOOP-9347. Add instructions to BUILDING.txt describing how to
-  build on Windows. (Chris Nauroth via suresh)
-
-  HADOOP-9348. Address TODO in winutils to add more command line usage
-  and examples. (Chris Nauroth via suresh)
-
-  HADOOP-9354. Windows native project files missing license headers.
-  (Chris Nauroth via suresh)
-
-  HADOOP-9356. Remove remaining references to cygwin/cygpath from scripts.
-  (Chris Nauroth via suresh)
-
-  HADOOP-9232. JniBasedUnixGroupsMappingWithFallback fails on Windows
-  with UnsatisfiedLinkError. (Ivan Mitic via suresh)
-
-  HADOOP-9368. Add timeouts to new tests in branch-trunk-win.
-  (Arpit Agarwal via suresh)
-
-Patch equivalent to trunk committed to branch-trunk-win
-
-  HADOOP-8924. Add maven plugin alternative to shell script to save
-  package-info.java. (Chris Nauroth via suresh)
-

hadoop-common-project/hadoop-common/CHANGES.txt

@@ -28,6 +28,10 @@ Release 2.0.5-beta - UNRELEASED
     HADOOP-9338. FsShell Copy Commands Should Optionally Preserve File
     Attributes. (Nick White via atm)
 
+    HADOOP-8562. Enhancements to support Hadoop on Windows Server and Windows
+    Azure environments. (See breakdown of tasks below for subtasks and
+    contributors)
+
   IMPROVEMENTS
 
     HADOOP-9253. Capture ulimit info in the logs at service start time.
@@ -191,6 +195,115 @@ Release 2.0.5-beta - UNRELEASED
     (which in turn throws SIGABRT) causing client crashes. (Colin Patrick
     McCabe via atm)
 
+  BREAKDOWN OF HADOOP-8562 SUBTASKS
+
+    HADOOP-8924. Hadoop Common creating package-info.java must not depend on
+    sh. (Chris Nauroth via suresh)
+
+    HADOOP-8945. Merge winutils from branch-1-win to branch-trunk-win.
+    (Bikas Saha, Chuan Liu, Giridharan Kesavan, Ivan Mitic, and Steve Maine
+    ported by Chris Nauroth via suresh)
+
+    HADOOP-8946. winutils: compile codebase during Maven build on 
+    branch-trunk-win. (Chris Nauroth via suresh)
+
+    HADOOP-8947. Merge FileUtil and Shell changes from branch-1-win to 
+    branch-trunk-win to enable initial test pass. (Raja Aluri, Davio Lao, 
+    Sumadhur Reddy Bolli, Ahmed El Baz, Kanna Karanam, Chuan Liu, 
+    Ivan Mitic, Chris Nauroth, and Bikas Saha via suresh)
+
+    HADOOP-8954. "stat" executable not found on Windows. (Bikas Saha, Ivan Mitic
+    ported by Chris Narouth via suresh)
+
+    HADOOP-8959. TestUserGroupInformation fails on Windows due to "id" executable
+    not found. (Bikas Saha, Ivan Mitic, ported by Chris Narouth via suresh)
+
+    HADOOP-8955. "chmod" executable not found on Windows.
+    (Chris Nauroth via suresh)
+
+    HADOOP-8960. TestMetricsServlet fails on Windows. (Ivan Mitic via suresh)
+
+    HADOOP-8961. GenericOptionsParser URI parsing failure on Windows.
+    (Ivan Mitic via suresh)
+
+    HADOOP-8949. Remove FileUtil.CygPathCommand dead code. (Chris Nauroth via 
+    suresh)
+
+    HADOOP-8956. FileSystem.primitiveMkdir failures on Windows cause multiple
+    test suites to fail. (Chris Nauroth via suresh)
+
+    HADOOP-8978. TestTrash fails on Windows. (Chris Nauroth via suresh)
+
+    HADOOP-8979. TestHttpServer fails on Windows. (Chris Nauroth via suresh)
+
+    HADOOP-8953. Shell PathData parsing failures on Windows. (Arpit Agarwal via
+    suresh)
+
+    HADOOP-8975. TestFileContextResolveAfs fails on Windows. (Chris Nauroth via
+    suresh)
+
+    HADOOP-8977. Multiple FsShell test failures on Windows. (Chris Nauroth via
+    suresh)
+
+    HADOOP-9005. Merge hadoop cmd line scripts from branch-1-win. (David Lao, 
+    Bikas Saha, Lauren Yang, Chuan Liu, Thejas M Nair and Ivan Mitic via suresh)
+
+    HADOOP-9008. Building hadoop tarball fails on Windows. (Chris Nauroth via
+    suresh)
+
+    HADOOP-9011. saveVersion.py does not include branch in version annotation.
+    (Chris Nauroth via suresh)
+
+    HADOOP-9110. winutils ls off-by-one error indexing MONTHS array can cause 
+    access violation. (Chris Nauroth via suresh)
+
+    HADOOP-9056. Build native library on Windows. (Chuan Liu, Arpit Agarwal via
+    suresh)
+
+    HADOOP-9144. Fix findbugs warnings. (Chris Nauroth via suresh)
+
+    HADOOP-9081. Add TestWinUtils. (Chuan Liu, Ivan Mitic, Chris Nauroth, 
+    and Bikas Saha via suresh)
+
+    HADOOP-9146. Fix sticky bit regression on branch-trunk-win.
+    (Chris Nauroth via suresh)
+
+    HADOOP-9266. Fix javac, findbugs, and release audit warnings on
+    branch-trunk-win. (Chris Nauroth via suresh)
+
+    HADOOP-9270. Remove a stale java comment from FileUtil. (Chris Nauroth via
+    szetszwo)
+
+    HADOOP-9271. Revert Python build scripts from branch-trunk-win.
+    (Chris Nauroth via suresh)
+
+    HADOOP-9313. Remove spurious mkdir from hadoop-config.cmd.
+    (Ivan Mitic via suresh)
+
+    HADOOP-9309. Test failures on Windows due to UnsatisfiedLinkError
+    in NativeCodeLoader#buildSupportsSnappy. (Arpit Agarwal via suresh)
+
+    HADOOP-9347. Add instructions to BUILDING.txt describing how to
+    build on Windows. (Chris Nauroth via suresh)
+
+    HADOOP-9348. Address TODO in winutils to add more command line usage
+    and examples. (Chris Nauroth via suresh)
+
+    HADOOP-9354. Windows native project files missing license headers.
+    (Chris Nauroth via suresh)
+
+    HADOOP-9356. Remove remaining references to cygwin/cygpath from scripts.
+    (Chris Nauroth via suresh)
+
+    HADOOP-9232. JniBasedUnixGroupsMappingWithFallback fails on Windows
+    with UnsatisfiedLinkError. (Ivan Mitic via suresh)
+
+    HADOOP-9368. Add timeouts to new tests in branch-trunk-win.
+    (Arpit Agarwal via suresh)
+
+    HADOOP-9373. Merge CHANGES.branch-trunk-win.txt to CHANGES.txt.
+    (suresh)
+
 Release 2.0.4-alpha - 2013-04-25 
 
   INCOMPATIBLE CHANGES

hadoop-common-project/hadoop-common/pom.xml

@@ -415,7 +415,6 @@
         <configuration>
           <excludes>
             <exclude>CHANGES.txt</exclude>
-            <exclude>CHANGES.branch-trunk-win.txt</exclude>
             <exclude>.idea/**</exclude>
             <exclude>src/main/conf/*</exclude>
             <exclude>src/main/docs/**</exclude>

hadoop-hdfs-project/hadoop-hdfs/CHANGES.branch-trunk-win.txt

@@ -1,13 +0,0 @@
-branch-trunk-win changes - unreleased
-
-  HDFS-4145. Merge hdfs cmd line scripts from branch-1-win. (David Lao,
-  Bikas Saha, Lauren Yang, Chuan Liu, Thejas M Nair and Ivan Mitic via suresh)
-
-  HDFS-4163. HDFS distribution build fails on Windows. (Chris Nauroth via
-  suresh)
-
-  HDFS-4316. branch-trunk-win contains test code accidentally added during 
-  work on fixing tests on Windows. (Chris Nauroth via suresh)
-
-  HDFS-4297. Fix issues related to datanode concurrent reading and writing on
-  Windows. (Arpit Agarwal, Chuan Liu via suresh)

hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

@@ -18,6 +18,10 @@ Release 2.0.5-beta - UNRELEASED
 
     HDFS-4334. Add a unique id to INode.  (Brandon Li via szetszwo)
 
+    HADOOP-8562. Enhancements to support Hadoop on Windows Server and Windows
+    Azure environments. (See breakdown of tasks below for subtasks and
+    contributors)
+    
     HDFS-4339. Persist inode id in fsimage and editlog. (Brandon Li via
     suresh)
 
@@ -329,6 +333,20 @@ Release 2.0.5-beta - UNRELEASED
     HDFS-4661. A few little code cleanups of some HDFS-347-related code. (Colin
     Patrick McCabe via atm)
 
+  BREAKDOWN OF HADOOP-8562 SUBTASKS
+
+    HDFS-4145. Merge hdfs cmd line scripts from branch-1-win. (David Lao,
+    Bikas Saha, Lauren Yang, Chuan Liu, Thejas M Nair and Ivan Mitic via suresh)
+
+    HDFS-4163. HDFS distribution build fails on Windows. (Chris Nauroth via
+    suresh)
+
+    HDFS-4316. branch-trunk-win contains test code accidentally added during 
+    work on fixing tests on Windows. (Chris Nauroth via suresh)
+
+    HDFS-4297. Fix issues related to datanode concurrent reading and writing on
+    Windows. (Arpit Agarwal, Chuan Liu via suresh)
+
 Release 2.0.4-alpha - 2013-04-25
 
   INCOMPATIBLE CHANGES

hadoop-hdfs-project/hadoop-hdfs/pom.xml

@@ -510,8 +510,6 @@ http://maven.apache.org/xsd/maven-4.0.0.xsd">
           <excludes>
             <exclude>CHANGES.txt</exclude>
             <exclude>CHANGES.HDFS-1623.txt</exclude>
-            <exclude>CHANGES.HDFS-347.txt</exclude>
-            <exclude>CHANGES.branch-trunk-win.txt</exclude>
             <exclude>.idea/**</exclude>
             <exclude>src/main/conf/*</exclude>
             <exclude>src/main/docs/**</exclude>

hadoop-mapreduce-project/CHANGES.branch-trunk-win.txt

@@ -1,17 +0,0 @@
-branch-trunk-win changes - unreleased
-
-  MAPREDUCE-4739. Some MapReduce tests fail to find winutils.
-  (Chris Nauroth via suresh)
-
-  MAPREDUCE-4780. MapReduce distribution build fails on Windows.
-  (Chris Nauroth via suresh)
-
-  MAPREDUCE-4790. MapReduce build script would be more readable using abspath.
-  (Chris Nauroth via suresh)
-
-  MAPREDUCE-4869. Fix TestMapReduceChildJVM. (Chris Nauroth via acmurthy)
-
-  MAPREDUCE-4870. Fix TestMRJobsWithHistoryService. (Chris Nauroth via acmurthy)
-
-  MAPREDUCE-4983. Fixed various platform specific assumptions in various tests,
-  so that they can pass on Windows too. (Chris Nauroth via vinodkv)

hadoop-mapreduce-project/CHANGES.txt

@@ -21,6 +21,10 @@ Release 2.0.5-beta - UNRELEASED
 
   NEW FEATURES
 
+    HADOOP-8562. Enhancements to support Hadoop on Windows Server and Windows
+    Azure environments. (See breakdown of tasks below for subtasks and
+    contributors)
+
   IMPROVEMENTS
 
     MAPREDUCE-3008. Improvements to cumulative CPU emulation for short running
@@ -284,6 +288,24 @@ Release 2.0.5-beta - UNRELEASED
     MAPREDUCE-5261. Fix issues in TestRMContainerAllocator after YARN-617.
     (Omkar Vinit Joshi via vinodkv)
 
+  BREAKDOWN OF HADOOP-8562 SUBTASKS
+
+    MAPREDUCE-4739. Some MapReduce tests fail to find winutils.
+    (Chris Nauroth via suresh)
+
+    MAPREDUCE-4780. MapReduce distribution build fails on Windows.
+    (Chris Nauroth via suresh)
+
+    MAPREDUCE-4790. MapReduce build script would be more readable using abspath.
+    (Chris Nauroth via suresh)
+
+    MAPREDUCE-4869. Fix TestMapReduceChildJVM. (Chris Nauroth via acmurthy)
+
+    MAPREDUCE-4870. Fix TestMRJobsWithHistoryService. (Chris Nauroth via acmurthy)
+
+    MAPREDUCE-4983. Fixed various platform specific assumptions in various tests,
+    so that they can pass on Windows too. (Chris Nauroth via vinodkv)
+
 Release 2.0.4-alpha - 2013-04-25
 
   INCOMPATIBLE CHANGES

hadoop-mapreduce-project/pom.xml

@@ -210,7 +210,6 @@
           <excludes>
             <exclude>.eclipse.templates/</exclude>
             <exclude>CHANGES.txt</exclude>
-            <exclude>CHANGES.branch-trunk-win.txt</exclude>
             <exclude>lib/jdiff/**</exclude>
           </excludes>
         </configuration>

hadoop-yarn-project/CHANGES.branch-trunk-win.txt

@@ -1,29 +0,0 @@
-branch-trunk-win changes - unreleased
-
-  YARN-158. Yarn creating package-info.java must not depend on sh.
-  (Chris Nauroth via suresh)
-
-  YARN-176. Some YARN tests fail to find winutils. (Chris Nauroth via suresh)
-  
-  YARN-207. YARN distribution build fails on Windows. (Chris Nauroth via
-  suresh)
-
-  YARN-199. Yarn cmd line scripts for windows. (Ivan Mitic via suresh)
-
-  YARN-213. YARN build script would be more readable using abspath.
-  (Chris Nauroth via suresh)
-
-  YARN-233. Added support for running containers in MS Windows to YARN. (Chris
-  Nauroth via acmurthy)
-
-  YARN-234. Added support for process tree and resource calculator in MS Windows 
-  to YARN. (Chris Nauroth via acmurthy)
-
-  YARN-259. Fix LocalDirsHandlerService to use Path rather than URIs. (Xuan
-  Gong via acmurthy) 
-
-  YARN-316. YARN container launch may exceed maximum Windows command line 
-  length due to long classpath. (Chris Nauroth via suresh)
-
-  YARN-359. Fixing commands for container signalling in Windows. (Chris Nauroth
-  via vinodkv)

hadoop-yarn-project/CHANGES.txt

@@ -60,6 +60,10 @@ Release 2.0.5-beta - UNRELEASED
     YARN-563. Add the concept of an application-type for each application.
     (Mayank Bansal via vinodkv)
 
+    HADOOP-8562. Enhancements to support Hadoop on Windows Server and Windows
+    Azure environments. (See breakdown of tasks below for subtasks and
+    contributors)
+
   IMPROVEMENTS
 
     YARN-365. Change NM heartbeat handling to not generate a scheduler event
@@ -332,6 +336,37 @@ Release 2.0.5-beta - UNRELEASED
     YARN-715. Fixed unit test failures - TestDistributedShell and
     TestUnmanagedAMLauncher. (Vinod Kumar Vavilapalli via sseth)
 
+  BREAKDOWN OF HADOOP-8562 SUBTASKS
+
+    YARN-158. Yarn creating package-info.java must not depend on sh.
+    (Chris Nauroth via suresh)
+
+    YARN-176. Some YARN tests fail to find winutils. (Chris Nauroth via suresh)
+    
+    YARN-207. YARN distribution build fails on Windows. (Chris Nauroth via
+    suresh)
+
+    YARN-199. Yarn cmd line scripts for windows. (Ivan Mitic via suresh)
+
+    YARN-213. YARN build script would be more readable using abspath.
+    (Chris Nauroth via suresh)
+
+    YARN-233. Added support for running containers in MS Windows to YARN. (Chris
+    Nauroth via acmurthy)
+
+    YARN-234. Added support for process tree and resource calculator in MS Windows 
+    to YARN. (Chris Nauroth via acmurthy)
+
+    YARN-259. Fix LocalDirsHandlerService to use Path rather than URIs. (Xuan
+    Gong via acmurthy) 
+
+    YARN-316. YARN container launch may exceed maximum Windows command line 
+    length due to long classpath. (Chris Nauroth via suresh)
+
+    YARN-359. Fixing commands for container signalling in Windows. (Chris Nauroth
+    via vinodkv)
+
+
 Release 2.0.4-alpha - 2013-04-25 
 
   INCOMPATIBLE CHANGES

hadoop-yarn-project/pom.xml

@@ -197,7 +197,6 @@
         <configuration>
           <excludes>
             <exclude>CHANGES.txt</exclude>
-            <exclude>CHANGES.branch-trunk-win.txt</exclude>
           </excludes>
         </configuration>
       </plugin>