HADOOP-3168. Reduce the amount of logging in streaming to an

exponentially increasing number of records (up to 10,000
records/log). Contributed by Zheng Shao.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@644900 13f79535-47bb-0310-9956-ffa450edef68

CHANGES.txt

@@ -249,6 +249,10 @@ Trunk (unreleased changes)
     HADOOP-3140. Doesn't add a task in the commit queue if the task hadn't
     generated any output. (Amar Kamat via ddas)
 
+    HADOOP-3168. Reduce the amount of logging in streaming to an
+    exponentially increasing number of records (up to 10,000
+    records/log). (Zheng Shao via omalley)
+ 
   BUG FIXES
 
     HADOOP-2195. '-mkdir' behaviour is now closer to Linux shell in case of

src/contrib/streaming/src/java/org/apache/hadoop/streaming/PipeMapRed.java

@@ -511,8 +511,11 @@ public abstract class PipeMapRed {
       String info = numRecInfo();
       logprintln(info);
       logflush();
-      //nextRecReadLog_ *= 10;
-      nextRecReadLog_ += 100;
+      if (nextRecReadLog_ < 100000) {
+	  nextRecReadLog_ *= 10;
+      } else {
+	  nextRecReadLog_ += 100000;
+      }
     }
   }