Преглед на файлове

MAPREDUCE-6257. Document encrypted spills (Bibin A Chundatt via aw)

Allen Wittenauer преди 10 години
родител
ревизия
fb1be0b310

+ 2 - 0
hadoop-mapreduce-project/CHANGES.txt

@@ -206,6 +206,8 @@ Trunk (Unreleased)
     MAPREDUCE-6406. Update FileOutputCommitter.FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT
     to match mapred-default.xml. (Ray Chiang via devaraj)
 
+    MAPREDUCE-6257. Document encrypted spills (Bibin A Chundatt via aw)
+
   BREAKDOWN OF MAPREDUCE-2841 (NATIVE TASK) SUBTASKS
 
     MAPREDUCE-5985. native-task: Fix build on macosx. Contributed by

+ 20 - 0
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml

@@ -1760,4 +1760,24 @@
   </description>
 </property>
 
+<property>
+  <name>mapreduce.job.encrypted-intermediate-data</name>
+  <value>false</value>
+  <description>Encrypt intermediate MapReduce spill files or not
+  default is false</description>
+</property>
+
+<property>
+  <name>mapreduce.job.encrypted-intermediate-data-key-size-bits</name>
+  <value>128</value>
+  <description>Mapreduce encrypt data key size default is 128</description>
+</property>
+
+<property>
+  <name>mapreduce.job.encrypted-intermediate-data.buffer.kb</name>
+  <value>128</value>
+  <description>Buffer size for intermediate encrypt data in kb
+  default is 128</description>
+</property>
+
 </configuration>

+ 7 - 1
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/EncryptedShuffle.md

@@ -260,4 +260,10 @@ Encrypted Intermediate Data Spill files
 This capability allows encryption of the intermediate files generated during the merge and shuffle phases.
 It can be enabled by setting the `mapreduce.job.encrypted-intermediate-data` job property to `true`.
 
-**NOTE:** Currently, enabling encrypted intermediate data spills would restrict the number of attempts of the job to 1.
+|               Name |  Type |                                                           Description |
+|:---- |:---- |:---- |
+| mapreduce.job.encrypted-intermediate-data | boolean | Enable or disable encrypt intermediate mapreduce spill files.Default is false. |
+| mapreduce.job.encrypted-intermediate-data-key-size-bits | int | The key length used by keygenerator to encrypt data spilled to disk. |
+| mapreduce.job.encrypted-intermediate-data.buffer.kb | int | The buffer size in kb for stream written to disk after encryption. |
+
+**NOTE:** Currently, enabling encrypted intermediate data spills would restrict the number of attempts of the job to 1.