Browse Source

Update install instructions with svn unsplit (Thomas Graves via mahadev)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-279@1136111 13f79535-47bb-0310-9956-ffa450edef68
Mahadev Konar 14 years ago
parent
commit
6833585b22
3 changed files with 27 additions and 14 deletions
  1. 2 0
      mapreduce/CHANGES.txt
  2. 24 13
      mapreduce/INSTALL
  3. 1 1
      mapreduce/yarn/README

+ 2 - 0
mapreduce/CHANGES.txt

@@ -4,6 +4,8 @@ Trunk (unreleased changes)
 
 
     MAPREDUCE-279
+   
+    Update install instructions with svn unsplit (Thomas Graves via mahadev)
 
     MAPREDUCE-2587. Generate yarn version for UI. (Thomas Graves via llu)
 

+ 24 - 13
mapreduce/INSTALL

@@ -1,18 +1,28 @@
 To compile  Hadoop Mapreduce next following, do the following:
 
-Step 1) Download Hadoop Common
+Step 1) Install dependencies for yarn
 
-svn checkout http://svn.apache.org/repos/asf/hadoop/common/branches/yahoo-merge/
+See http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/yarn/README
+Make sure protbuf library is in your library path or set: export LD_LIBRARY_PATH=/usr/local/lib
+
+Step 2) Checkout
+
+svn checkout http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/
+
+Step 3) Build common
+
+Go to common directory
 ant veryclean mvn-install 
 
-Step 2) Download Hadoop HDFS 
+Step 4) Build HDFS 
 
-svn checkout http://svn.apache.org/repos/asf/hadoop/hdfs/branches/HDFS-1052/
+Go to hdfs directory
 ant veryclean mvn-install -Dresolvers=internal 
 
-Step 3) Go to the root directory of hadoop mapreduce
+Step 5) Build yarn and mapreduce
 
-Step 4) Run 
+Go to mapreduce directory
+export MAVEN_OPTS=-Xmx512m
 
 mvn clean install assembly:assembly
 ant veryclean jar jar-test  -Dresolvers=internal 
@@ -25,14 +35,14 @@ ant veryclean jar jar-test  -Dresolvers=internal
 You will see a tarball in
 ls target/hadoop-mapreduce-1.0-SNAPSHOT-bin.tar.gz  
 
-Step 5) Untar the tarball in a clean and different directory.
+Step 6) Untar the tarball in a clean and different directory.
 say HADOOP_YARN_INSTALL
 
 To run Hadoop Mapreduce next applications :
 
-Step 6) cd $HADOOP_YARN_INSTALL
+Step 7) cd $HADOOP_YARN_INSTALL
 
-Step 7) export the following variables:
+Step 8) export the following variables:
 
 HADOOP_MAPRED_HOME=
 HADOOP_COMMON_HOME=
@@ -41,11 +51,11 @@ YARN_HOME=directory where you untarred yarn
 HADOOP_CONF_DIR=
 YARN_CONF_DIR=$HADOOP_CONF_DIR
 
-Step 8) bin/yarn-daemon.sh start resourcemanager
+Step 9) bin/yarn-daemon.sh start resourcemanager
 
-Step 9) bin/yarn-daemon.sh start nodemanager
+Step 10) bin/yarn-daemon.sh start nodemanager
 
-Step 10) Create the following symlinks in hadoop-common/lib 
+Step 11) Create the following symlinks in hadoop-common/lib 
 
 ln -s $HADOOP_YARN_INSTALL/modules/hadoop-mapreduce-client-app-1.0-SNAPSHOT.jar .	
 ln -s $HADOOP_YARN_INSTALL/modules/yarn-api-1.0-SNAPSHOT.jar .
@@ -53,9 +63,10 @@ ln -s $HADOOP_YARN_INSTALL/modules/hadoop-mapreduce-client-common-1.0-SNAPSHOT.j
 ln -s $HADOOP_YARN_INSTALL/modules/yarn-common-1.0-SNAPSHOT.jar .
 ln -s $HADOOP_YARN_INSTALL/modules/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar .	
 ln -s $HADOOP_YARN_INSTALL/modules/yarn-server-common-1.0-SNAPSHOT.jar .
+ln -s $HADOOP_YARN_INSTALL/modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar .
 ln -s $HADOOP_YARN_INSTALL/lib/protobuf-java-2.4.0a.jar .
 
-Step 11) You are all set, an example on how to run a job is:
+Step 12) You are all set, an example on how to run a job is:
 
 $HADOOP_COMMON_HOME/bin/hadoop jar $HADOOP_MAPRED_HOME/build/hadoop-mapred-examples-0.22.0-SNAPSHOT.jar randomwriter -Dmapreduce.job.user.name=$USER -Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.blocksize=536870912 -Ddfs.block.size=536870912 -libjars $HADOOP_YARN_INSTALL/hadoop-mapreduce-1.0-SNAPSHOT/modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar output 
 

+ 1 - 1
mapreduce/yarn/README

@@ -11,7 +11,7 @@ Setup
 Install the Avro maven plugin in local maven repository:
 mvn install:install-file -Dfile=./avro-maven-plugin/avro-maven-plugin-1.4.0-SNAPSHOT.jar -DpomFile=./avro-maven-plugin/avro-maven-plugin-1.4.0-SNAPSHOT.pom
 
-Install protobuf 2.4.0a (Download from http://code.google.com/p/protobuf/downloads/list)
+Install protobuf 2.4.0a or higher (Download from http://code.google.com/p/protobuf/downloads/list)
  - install the protoc executable (configure, make, make install)
  - install the maven artifact (cd java; mvn install)
 Installing protoc requires gcc 4.1.x or higher.