소스 검색

HADOOP-5146. Fixes a race condition that causes LocalDirAllocator to miss files. Contributed by Devaraj Das.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.19@748785 13f79535-47bb-0310-9956-ffa450edef68
Hemanth Yamijala 16 년 전
부모
커밋
ed531e3f56
2개의 변경된 파일7개의 추가작업 그리고 11개의 파일을 삭제
  1. 3 0
      CHANGES.txt
  2. 4 11
      src/mapred/org/apache/hadoop/mapred/TaskRunner.java

+ 3 - 0
CHANGES.txt

@@ -29,6 +29,9 @@ Release 0.19.2 - Unreleased
     job is not in memory and a tasktracker comes to the jobtracker with a status
     report of a task belonging to that job. (Amar Kamat via ddas)
 
+    HADOOP-5146. Fixes a race condition that causes LocalDirAllocator to miss
+    files.  (Devaraj Das via yhemanth)
+
 Release 0.19.1 - 2009-02-23
 
   INCOMPATIBLE CHANGES

+ 4 - 11
src/mapred/org/apache/hadoop/mapred/TaskRunner.java

@@ -139,13 +139,9 @@ abstract class TaskRunner extends Thread {
             String cacheId = DistributedCache.makeRelative(archives[i],conf);
             String cachePath = TaskTracker.getCacheSubdir() + 
                                  Path.SEPARATOR + cacheId;
-            if (lDirAlloc.ifExists(cachePath, conf)) {
-              localPath =  lDirAlloc.getLocalPathToRead(cachePath, conf);
-            }
-            else {
-              localPath = lDirAlloc.getLocalPathForWrite(cachePath,
+            
+            localPath = lDirAlloc.getLocalPathForWrite(cachePath,
                                       fileStatus.getLen(), conf);
-            }
             baseDir = localPath.toString().replace(cacheId, "");
             p[i] = DistributedCache.getLocalCache(archives[i], conf, 
                                                   new Path(baseDir),
@@ -169,12 +165,9 @@ abstract class TaskRunner extends Thread {
             String cacheId = DistributedCache.makeRelative(files[i], conf);
             String cachePath = TaskTracker.getCacheSubdir() +
                                  Path.SEPARATOR + cacheId;
-            if (lDirAlloc.ifExists(cachePath,conf)) {
-              localPath =  lDirAlloc.getLocalPathToRead(cachePath, conf);
-            } else {
-              localPath = lDirAlloc.getLocalPathForWrite(cachePath,
+            
+            localPath = lDirAlloc.getLocalPathForWrite(cachePath,
                                       fileStatus.getLen(), conf);
-            }
             baseDir = localPath.toString().replace(cacheId, "");
             p[i] = DistributedCache.getLocalCache(files[i], conf, 
                                                   new Path(baseDir),