MAPREDUCE-2645. Updates to MRv2 INSTALL and README documentation. Contributed by Josh Wills.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-279@1143803 13f79535-47bb-0310-9956-ffa450edef68

mapreduce/CHANGES.txt

@@ -5,6 +5,9 @@ Trunk (unreleased changes)
 
     MAPREDUCE-279
 
+    MAPREDUCE-2645. Updates to MRv2 INSTALL and README documentation.
+    (Josh Wills via vinodkv)
+
     MAPREDUCE-2633. Add a getCounter(Enum) method to the Counters record. 
     (Josh Wills via sharad)
 

mapreduce/INSTALL

@@ -29,7 +29,7 @@ ant veryclean jar jar-test  -Dresolvers=internal
 
 In case you want to skip the tests run:
 
-mvn clean install assembly:assembly -Dmaven.test.skip.exec=true
+mvn clean install assembly:assembly -DskipTests
 ant veryclean jar jar-test  -Dresolvers=internal 
 
 You will see a tarball in
@@ -55,7 +55,9 @@ Step 9) bin/yarn-daemon.sh start resourcemanager
 
 Step 10) bin/yarn-daemon.sh start nodemanager
 
-Step 11) Create the following symlinks in hadoop-common/lib 
+Step 11) bin/yarn-daemon.sh start historyserver
+
+Step 12) Create the following symlinks in hadoop-common/lib 
 
 ln -s $HADOOP_YARN_INSTALL/modules/hadoop-mapreduce-client-app-1.0-SNAPSHOT.jar .	
 ln -s $HADOOP_YARN_INSTALL/modules/yarn-api-1.0-SNAPSHOT.jar .
@@ -66,8 +68,22 @@ ln -s $HADOOP_YARN_INSTALL/modules/yarn-server-common-1.0-SNAPSHOT.jar .
 ln -s $HADOOP_YARN_INSTALL/modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar .
 ln -s $HADOOP_YARN_INSTALL/lib/protobuf-java-2.4.0a.jar .
 
-Step 12) You are all set, an example on how to run a job is:
+Step 13) Yarn daemons are up! But for running mapreduce applications, which now are in user land, you need to setup nodemanager with the following configuration in your yarn-site.xml before you start the nodemanager.
+    <property>
+      <name>nodemanager.auxiluary.services</name>
+      <value>mapreduce.shuffle</value>
+    </property>
+
+    <property>
+      <name>nodemanager.aux.service.mapreduce.shuffle.class</name>
+      <value>org.apache.hadoop.mapred.ShuffleHandler</value>
+    </property>
+
+Step 14) You are all set, an example on how to run a mapreduce job is:
 
-$HADOOP_COMMON_HOME/bin/hadoop jar $HADOOP_MAPRED_HOME/build/hadoop-mapred-examples-0.22.0-SNAPSHOT.jar randomwriter -Dmapreduce.job.user.name=$USER -Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.blocksize=536870912 -Ddfs.block.size=536870912 -libjars $HADOOP_YARN_INSTALL/hadoop-mapreduce-1.0-SNAPSHOT/modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar output 
+cd $HADOOP_MAPRED_HOME
+ant examples -Dresolvers=internal
+$HADOOP_COMMON_HOME/bin/hadoop jar $HADOOP_MAPRED_HOME/build/hadoop-mapred-examples-0.22.0-SNAPSHOT.jar randomwriter -Dmapreduce.job.user.name=$USER -Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory -Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.blocksize=536870912 -Ddfs.block.size=536870912 -libjars $HADOOP_YARN_INSTALL/hadoop-mapreduce-1.0-SNAPSHOT/modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar output 
 
+The output on the command line should be almost similar to what you see in the JT/TT setup (Hadoop 0.20/0.21)
 

mapreduce/yarn/README

@@ -8,9 +8,6 @@ Maven: Maven 2
 
 Setup
 -----
-Install the Avro maven plugin in local maven repository:
-mvn install:install-file -Dfile=./avro-maven-plugin/avro-maven-plugin-1.4.0-SNAPSHOT.jar -DpomFile=./avro-maven-plugin/avro-maven-plugin-1.4.0-SNAPSHOT.pom
-
 Install protobuf 2.4.0a or higher (Download from http://code.google.com/p/protobuf/downloads/list)
  - install the protoc executable (configure, make, make install)
  - install the maven artifact (cd java; mvn install)
@@ -72,32 +69,5 @@ yarn-common - Utilities which can be used by yarn clients and server
 
 yarn-server - Implementation of the yarn-api
 	yarn-server-common - APIs shared between resourcemanager and nodemanager
-	yarn-server-nodemanager
-	yarn-server-resourcemanager
-
-yarn-prototype - AppMaster (Prototype) to launch TTs via nodemanager
-
-yarn-mapreduce - Mapreduce specific parts
-	yarn-mapreduce-common - Mapreduce utilities and interfae shared between client and mapreduce AppMaster
-	yarn-mapreduce-client - Mapreduce client (uses JobClient)
-	yarn-mapreduce-app - Mapreduce AppMaster
-
-yarn-master-worker - master-worker AppMaster
-
-General instructions for starting Yarn and MapReduce
-----------------------------------------------------
-At yarn project root,
-
-	mvn clean install
-
-ResourceManager:
-	cd yarn-server/yarn-server-resourcemanager
-	mvn exec:java -Dexec.mainClass=org.apache.hadoop.yarn.server.resourcemanager.ResourceManager
-
-NodeManager:
-	cd yarn-server/yarn-server-nodemanager
-	mvn exec:java -Dexec.mainClass=org.apache.hadoop.yarn.server.nodemanager.NodeManager
-
-MRAppMaster
-	cd yarn-mapreduce/yarn-mapreduce-app
-	mvn exec:java -Dexec.mainClass=org.apache.hadoop.yarn.mapreduce.app.MRAppMaster -Dexec.args="<clusterts> <appid>"
+	yarn-server-nodemanager (TaskTracker replacement)
+	yarn-server-resourcemanager (JobTracker replacement)