浏览代码

MAPREDUCE-6257. Document encrypted spills (Bibin A Chundatt via aw)

Allen Wittenauer 10 年之前
父节点
当前提交
fb1be0b310

+ 2 - 0
hadoop-mapreduce-project/CHANGES.txt

@@ -206,6 +206,8 @@ Trunk (Unreleased)
     MAPREDUCE-6406. Update FileOutputCommitter.FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT
     MAPREDUCE-6406. Update FileOutputCommitter.FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT
     to match mapred-default.xml. (Ray Chiang via devaraj)
     to match mapred-default.xml. (Ray Chiang via devaraj)
 
 
+    MAPREDUCE-6257. Document encrypted spills (Bibin A Chundatt via aw)
+
   BREAKDOWN OF MAPREDUCE-2841 (NATIVE TASK) SUBTASKS
   BREAKDOWN OF MAPREDUCE-2841 (NATIVE TASK) SUBTASKS
 
 
     MAPREDUCE-5985. native-task: Fix build on macosx. Contributed by
     MAPREDUCE-5985. native-task: Fix build on macosx. Contributed by

+ 20 - 0
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml

@@ -1760,4 +1760,24 @@
   </description>
   </description>
 </property>
 </property>
 
 
+<property>
+  <name>mapreduce.job.encrypted-intermediate-data</name>
+  <value>false</value>
+  <description>Encrypt intermediate MapReduce spill files or not
+  default is false</description>
+</property>
+
+<property>
+  <name>mapreduce.job.encrypted-intermediate-data-key-size-bits</name>
+  <value>128</value>
+  <description>Mapreduce encrypt data key size default is 128</description>
+</property>
+
+<property>
+  <name>mapreduce.job.encrypted-intermediate-data.buffer.kb</name>
+  <value>128</value>
+  <description>Buffer size for intermediate encrypt data in kb
+  default is 128</description>
+</property>
+
 </configuration>
 </configuration>

+ 7 - 1
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/EncryptedShuffle.md

@@ -260,4 +260,10 @@ Encrypted Intermediate Data Spill files
 This capability allows encryption of the intermediate files generated during the merge and shuffle phases.
 This capability allows encryption of the intermediate files generated during the merge and shuffle phases.
 It can be enabled by setting the `mapreduce.job.encrypted-intermediate-data` job property to `true`.
 It can be enabled by setting the `mapreduce.job.encrypted-intermediate-data` job property to `true`.
 
 
-**NOTE:** Currently, enabling encrypted intermediate data spills would restrict the number of attempts of the job to 1.
+|               Name |  Type |                                                           Description |
+|:---- |:---- |:---- |
+| mapreduce.job.encrypted-intermediate-data | boolean | Enable or disable encrypt intermediate mapreduce spill files.Default is false. |
+| mapreduce.job.encrypted-intermediate-data-key-size-bits | int | The key length used by keygenerator to encrypt data spilled to disk. |
+| mapreduce.job.encrypted-intermediate-data.buffer.kb | int | The buffer size in kb for stream written to disk after encryption. |
+
+**NOTE:** Currently, enabling encrypted intermediate data spills would restrict the number of attempts of the job to 1.