HADOOP-2486. When the list of files from the InMemoryFileSystem is obtained for merging, this patch will ensure that only those files whose checksums have also got created (renamed) are returned. Contributed by Devaraj Das.

git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk@606854 13f79535-47bb-0310-9956-ffa450edef68

CHANGES.txt

@@ -333,6 +333,10 @@ Branch 0.15 (unreleased)
     round-robin disk selections randomly. This helps in spreading data across
     multiple partitions much better. (acmurhty)
 
+    HADOOP-2486. When the list of files from the InMemoryFileSystem is obtained
+    for merging, this patch will ensure that only those files whose checksums
+    have also got created (renamed) are returned. (ddas)
+
   IMPROVEMENTS
 
     HADOOP-2160.  Remove project-level, non-user documentation from

src/java/org/apache/hadoop/mapred/ReduceTask.java

@@ -1386,8 +1386,11 @@ class ReduceTask extends Task {
       public void run() {
         LOG.info(reduceTask.getTaskId() + " Thread started: " + getName());
         try {
+          Path[] inMemClosedFiles;
           //initiate merge
-          Path[] inMemClosedFiles = inMemFileSys.getFiles(MAP_OUTPUT_FILTER);
+          synchronized (ReduceTask.this) {
+            inMemClosedFiles = inMemFileSys.getFiles(MAP_OUTPUT_FILTER);
+          }
           //Note that the above Path[] could be of length 0 if all copies are 
           //in flight. So we make sure that we have some 'closed' map
           //output files to merge to get the benefit of in-memory merge