HADOOP-1152. Fix race condition in MapOutputCopier.copyOutput file rename causing possible reduce task hang. Contributed by Tahir Hashmi.

git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk@531084 13f79535-47bb-0310-9956-ffa450edef68

CHANGES.txt

@@ -234,6 +234,10 @@ Trunk (unreleased changes)
 70. HADOOP-1275.  Fix misspelled job notification property in
     hadoop-default.xml.  (Alejandro Abdelnur via tomwhite)
 
+71. HADOOP-1152.  Fix race condition in MapOutputCopier.copyOutput file
+    rename causing possible reduce task hang.
+    (Tahir Hashmi via tomwhite)
+
 
 Release 0.12.3 - 2007-04-06
 

src/java/org/apache/hadoop/mapred/ReduceTask.java

@@ -698,14 +698,17 @@ class ReduceTask extends Task {
             fs.delete(tmpFilename);
             return CopyResult.OBSOLETE;
           }
+          
+          bytes = fs.getLength(tmpFilename);
           // if we can't rename the file, something is broken (and IOException
           // will be thrown). 
           if (!fs.rename(tmpFilename, finalFilename)) {
             fs.delete(tmpFilename);
+            bytes = -1;
             throw new IOException("failure to rename map output " + 
                                   tmpFilename);
           }
-          bytes = fs.getLength(finalFilename);
+          
           LOG.info(reduceId + " done copying " + loc.getMapTaskId() +
                    " output from " + loc.getHost() + ".");
           //Create a thread to do merges. Synchronize access/update to