Merge -r 606853:606854 from trunk to 0.15 branch. Fixes HADOOP-2486.

git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/branches/branch-0.15@606855 13f79535-47bb-0310-9956-ffa450edef68

CHANGES.txt

@@ -39,6 +39,10 @@ Branch 0.15 (unreleased)
     round-robin disk selections randomly. This helps in spreading data across
     multiple partitions much better. (acmurhty)
 
+    HADOOP-2486. When the list of files from the InMemoryFileSystem is obtained
+    for merging, this patch will ensure that only those files whose checksums
+    have also got created (renamed) are returned. (ddas)
+
   IMPROVEMENTS
 
     HADOOP-2160.  Remove project-level, non-user documentation from

src/java/org/apache/hadoop/mapred/ReduceTask.java

@@ -1290,8 +1290,11 @@ class ReduceTask extends Task {
       public void run() {
         LOG.info(reduceTask.getTaskId() + " Thread started: " + getName());
         try {
+          Path[] inMemClosedFiles;
           //initiate merge
-          Path[] inMemClosedFiles = inMemFileSys.getFiles(MAP_OUTPUT_FILTER);
+          synchronized (ReduceTask.this) {
+            inMemClosedFiles = inMemFileSys.getFiles(MAP_OUTPUT_FILTER);
+          }
           //Note that the above Path[] could be of length 0 if all copies are 
           //in flight. So we make sure that we have some 'closed' map
           //output files to merge to get the benefit of in-memory merge