HADOOP-7045. TestDU fails on systems with local file systems with

extended attributes. (eli)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security@1148088 13f79535-47bb-0310-9956-ffa450edef68

CHANGES.txt

@@ -1,6 +1,6 @@
 Hadoop Change Log
 
-Release 0.20.206.0 - unreleased
+Release 0.20.205.0 - unreleased
 
   NEW FEATURES
 
@@ -17,10 +17,25 @@ Release 0.20.206.0 - unreleased
     HDFS-1836. Thousand of CLOSE_WAIT socket. Contributed by Todd Lipcon,
     ported to security branch by Bharath Mundlapudi. (via mattf)
 
-Release 0.20.205.0 - unreleased
+Release 0.20.204.0 - unreleased
+
+  NEW FEATURES
+
+    HADOOP-6255. Create RPM and Debian packages for common. Changes deployment
+    layout to be consistent across the binary tgz, rpm, and deb. Adds setup
+    scripts for easy one node cluster configuration and user creation.
+    (Eric Yang via omalley)
+
+    HADOOP-7324. Ganglia plugins for metrics v2. (Priyo Mustafi via llu)
 
   BUG FIXES
 
+    HADOOP-7045. TestDU fails on systems with local file systems with 
+    extended attributes. (eli)
+
+    MAPREDUCE-2495. exit() the TaskTracker when the distributed cache cleanup
+    thread dies. (Robert Joseph Evans via cdouglas)
+
     HDFS-1878. TestHDFSServerPorts unit test failure - race condition 
     in FSNamesystem.close() causes NullPointerException without serious
     consequence. (mattf)
@@ -28,89 +43,57 @@ Release 0.20.205.0 - unreleased
     MAPREDUCE-2452. Moves the cancellation of delegation tokens to a separate
     thread. (ddas)
 
-    HDFS-1377. Quota bug for partial blocks allows quotas to be violated. (eli)
+    MAPREDUCE-2555. Avoid sprious logging from completedtasks. (Thomas Graves
+    via cdouglas)
 
     MAPREDUCE-2451. Log the details from health check script at the
     JobTracker. (Thomas Graves via cdouglas)
 
+    MAPREDUCE-2535. Fix NPE in JobClient caused by retirement. (Robert Joseph
+    Evans via cdouglas)
+
     MAPREDUCE-2456. Log the reduce taskID and associated TaskTrackers with
     failed fetch notifications in the JobTracker log.
     (Jeffrey Naisbitt via cdouglas)
 
+    HDFS-2044. TestQueueProcessingStatistics failing automatic test due to 
+    timing issues. (mattf)
+
     HADOOP-7248. Update eclipse target to generate .classpath from ivy config.
     (Thomas Graves and Tom White via cdouglas)
 
+    MAPREDUCE-2558. Add queue-level metrics 0.20-security branch (test fixes)
+    (Jeffrey Naisbitt via mahadev)
+    
+    HADOOP-7364. TestMiniMRDFSCaching fails if test.build.dir is set to 
+    something other than build/test. (Thomas Graves via mahadev)
+
     HADOOP-7277. Add generation of run configurations to eclipse target.
     (Jeffrey Naisbitt and Philip Zeyliger via cdouglas)
 
+    HADOOP-7373. Fix {start,stop}-{dfs,mapred} and hadoop-daemons.sh from
+    trying to use the wrong bin directory. (omalley)
+
     HADOOP-7274. Fix typos in IOUtils. (Jonathan Eagles via cdouglas)
 
+    HADOOP-7369. Fix permissions in tarball for sbin/* and libexec/* (omalley)
+
     MAPREDUCE-2479. Move distributed cache cleanup to a background task,
     backporting MAPREDUCE-1568. (Robert Joseph Evans via cdouglas)
 
-    HADOOP-7272. Remove unnecessary security related info logs. (suresh)
+    HADOOP-7356. Fix bin/hadoop scripts (eyang via omalley)
 
-    HDFS-1906. Remove logging exception stack trace in client logs when one of
-    the datanode targets to read from is not reachable. (suresh)
+    HADOOP-7272. Remove unnecessary security related info logs. (suresh)
 
     MAPREDUCE-2514. Fix typo in TaskTracker ReinitTrackerAction log message.
     (Jonathan Eagles via cdouglas)
 
+    HDFS-1906. Remove logging exception stack trace in client logs when one of
+    the datanode targets to read from is not reachable. (suresh)
+
     MAPREDUCE-2490. Add logging to graylist and blacklist activity to aid
     diagnosis of related issues. (Jonathan Eagles via cdouglas)
 
-    MAPREDUCE-2495. exit() the TaskTracker when the distributed cache cleanup
-    thread dies. (Robert Joseph Evans via cdouglas)
-
-    MAPREDUCE-2555. Avoid sprious logging from completedtasks. (Thomas Graves
-    via cdouglas)
-
-    MAPREDUCE-2558. Add JobTracker metrics for scheduling queues. (Jeffrey
-    Naisbitt via cdouglas)
-
-    MAPREDUCE-2535. Fix NPE in JobClient caused by retirement. (Robert Joseph
-    Evans via cdouglas)
-
-    HADOOP-7144. Expose JMX metrics via JSON servlet. (Robert Joseph Evans via
-    cdouglas)
-
-    MAPREDUCE-2524. Port reduce failure reporting semantics from trunk, to
-    fail faulty maps more aggressively. (Thomas Graves via cdouglas)
-
-    MAPREDUCE-2529. Add support for regex-based shuffle metric counting
-    exceptions. (Thomas Graves via cdouglas)
-
-    HDFS-2044. TestQueueProcessingStatistics failing automatic test due to 
-    timing issues. (mattf)
-
-    MAPREDUCE-2558. Add queue-level metrics 0.20-security branch (test fixes)
-    (Jeffrey Naisbitt via mahadev) 
-
-    HADOOP-7364. TestMiniMRDFSCaching fails if test.build.dir is set to 
-    something other than build/test (Thomas Graves via mahadev)
-
-Release 0.20.204.0 - unreleased
-
-  NEW FEATURES
-
-    HADOOP-6255. Create RPM and Debian packages for common. Changes deployment
-    layout to be consistent across the binary tgz, rpm, and deb. Adds setup
-    scripts for easy one node cluster configuration and user creation.
-    (Eric Yang via omalley)
-
-    HADOOP-7324. Ganglia plugins for metrics v2. (Priyo Mustafi via llu)
-
-  BUG FIXES
-
-    HADOOP-7373. Fix {start,stop}-{dfs,mapred} and hadoop-daemons.sh from
-    trying to use the wrong bin directory. (omalley)
-
-    HADOOP-7369. Fix permissions in tarball for sbin/* and libexec/* (omalley)
-
-    HADOOP-7356. Fix bin/hadoop scripts (eyang via omalley)
-
-    MAPREDUCE-2316. Updated CapacityScheduler documentation. (acmurthy) 
-
     MAPREDUCE-2447. Fix Child.java to set Task.jvmContext sooner to avoid
     corner cases in error handling. (Siddharth Seth via acmurthy) 
 
@@ -147,9 +130,6 @@ Release 0.20.204.0 - unreleased
     HDFS-1258. Clearing namespace quota on "/" corrupts fs image.  
     (Aaron T. Myers via szetszwo)
 
-    HADOOP-7215. RPC clients must use network interface corresponding to 
-    the host in the client's kerberos principal key. (suresh)
-
     HDFS-1189. Quota counts missed between clear quota and set quota.
     (John George via szetszwo)
 
@@ -166,11 +146,22 @@ Release 0.20.204.0 - unreleased
     by throwing an error to indicate the editlog needs to be empty.
     (suresh)
 
+    HDFS-1377. Quota bug for partial blocks allows quotas to be violated. (eli)
+
     HDFS-2057. Wait time to terminate the threads causes unit tests to
     take longer time. (Bharath Mundlapudi via suresh)
 
   IMPROVEMENTS
 
+    HADOOP-7144. Expose JMX metrics via JSON servlet. (Robert Joseph Evans via
+    cdouglas)
+
+    MAPREDUCE-2524. Port reduce failure reporting semantics from trunk, to
+    fail faulty maps more aggressively. (Thomas Graves via cdouglas)
+
+    MAPREDUCE-2529. Add support for regex-based shuffle metric counting
+    exceptions. (Thomas Graves via cdouglas)
+
     HADOOP-7398. Add mechanism to suppress warnings about use of HADOOP_HOME.
     (omalley)
 
@@ -217,6 +208,8 @@ Release 0.20.203.0 - 2011-5-11
 
     HADOOP-7143. Restore HadoopArchives. (Joep Rottinghuis via omalley)
 
+    MAPREDUCE-2316. Updated CapacityScheduler documentation. (acmurthy) 
+
     HADOOP-7243. Fix contrib unit tests missing dependencies. (omalley)
 
     MAPREDUCE-2355. Add a dampner to out-of-band heartbeats. (acmurthy) 
@@ -271,6 +264,9 @@ Release 0.20.203.0 - 2011-5-11
     HADOOP-6879. Provide SSH based (Jsch) remote execution API for system
     tests. (cos)
 
+    HADOOP-7215. RPC clients must use network interface corresponding to 
+    the host in the client's kerberos principal key. (suresh)
+
     HADOOP-7232. Fix Javadoc warnings. (omalley)
 
     HADOOP-7258. The Gzip codec should not return null decompressors. (omalley)

src/test/org/apache/hadoop/fs/TestDU.java

@@ -65,7 +65,10 @@ public class TestDU extends TestCase {
    * @throws InterruptedException
    */
   public void testDU() throws IOException, InterruptedException {
-    int writtenSize = 32*1024;   // writing 32K
+    final int writtenSize = 32*1024;   // writing 32K
+    // Allow for extra 4K on-disk slack for local file systems
+    // that may store additional file metadata (eg ext attrs).
+    final int slack = 4*1024;
     File file = new File(DU_DIR, "data");
     createFile(file, writtenSize);
 
@@ -76,7 +79,9 @@ public class TestDU extends TestCase {
     long duSize = du.getUsed();
     du.shutdown();
 
-    assertEquals(writtenSize, duSize);
+    assertTrue("Invalid on-disk size",
+        duSize >= writtenSize &&
+        writtenSize <= (duSize + slack));
     
     //test with 0 interval, will not launch thread 
     du = new DU(file, 0);
@@ -84,12 +89,16 @@ public class TestDU extends TestCase {
     duSize = du.getUsed();
     du.shutdown();
     
-    assertEquals(writtenSize, duSize);  
+    assertTrue("Invalid on-disk size",
+        duSize >= writtenSize &&
+        writtenSize <= (duSize + slack));
     
     //test without launching thread 
     du = new DU(file, 10000);
     duSize = du.getUsed();
-    
-    assertEquals(writtenSize, duSize);     
+
+    assertTrue("Invalid on-disk size",
+        duSize >= writtenSize &&
+        writtenSize <= (duSize + slack));
   }
 }