index.md.vm 26 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865
  1. <!---
  2. Licensed under the Apache License, Version 2.0 (the "License");
  3. you may not use this file except in compliance with the License.
  4. You may obtain a copy of the License at
  5. http://www.apache.org/licenses/LICENSE-2.0
  6. Unless required by applicable law or agreed to in writing, software
  7. distributed under the License is distributed on an "AS IS" BASIS,
  8. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  9. See the License for the specific language governing permissions and
  10. limitations under the License. See accompanying LICENSE file.
  11. -->
  12. #set ( $H3 = '###' )
  13. #set ( $H4 = '####' )
  14. #set ( $H5 = '#####' )
  15. Hadoop Key Management Server (KMS) - Documentation Sets
  16. =======================================================
  17. Hadoop KMS is a cryptographic key management server based on Hadoop's **KeyProvider** API.
  18. It provides a client and a server components which communicate over HTTP using a REST API.
  19. The client is a KeyProvider implementation interacts with the KMS using the KMS HTTP REST API.
  20. KMS and its client have built-in security and they support HTTP SPNEGO Kerberos authentication and HTTPS secure transport.
  21. KMS is a Java web-application and it runs using a pre-configured Tomcat bundled with the Hadoop distribution.
  22. KMS Client Configuration
  23. ------------------------
  24. The KMS client `KeyProvider` uses the **kms** scheme, and the embedded URL must be the URL of the KMS. For example, for a KMS running on `http://localhost:16000/kms`, the KeyProvider URI is `kms://http@localhost:16000/kms`. And, for a KMS running on `https://localhost:16000/kms`, the KeyProvider URI is `kms://https@localhost:16000/kms`
  25. KMS
  26. ---
  27. $H3 KMS Configuration
  28. Configure the KMS backing KeyProvider properties in the `etc/hadoop/kms-site.xml` configuration file:
  29. ```xml
  30. <property>
  31. <name>hadoop.kms.key.provider.uri</name>
  32. <value>jceks://file@/${user.home}/kms.keystore</value>
  33. </property>
  34. <property>
  35. <name>hadoop.security.keystore.java-keystore-provider.password-file</name>
  36. <value>kms.keystore.password</value>
  37. </property>
  38. ```
  39. The password file is looked up in the Hadoop's configuration directory via the classpath.
  40. NOTE: You need to restart the KMS for the configuration changes to take effect.
  41. $H3 KMS Cache
  42. KMS caches keys for short period of time to avoid excessive hits to the underlying key provider.
  43. The Cache is enabled by default (can be dissabled by setting the `hadoop.kms.cache.enable` boolean property to false)
  44. The cache is used with the following 3 methods only, `getCurrentKey()` and `getKeyVersion()` and `getMetadata()`.
  45. For the `getCurrentKey()` method, cached entries are kept for a maximum of 30000 millisecond regardless the number of times the key is being access (to avoid stale keys to be considered current).
  46. For the `getKeyVersion()` method, cached entries are kept with a default inactivity timeout of 600000 milliseconds (10 mins). This time out is configurable via the following property in the `etc/hadoop/kms-site.xml` configuration file:
  47. ```xml
  48. <property>
  49. <name>hadoop.kms.cache.enable</name>
  50. <value>true</value>
  51. </property>
  52. <property>
  53. <name>hadoop.kms.cache.timeout.ms</name>
  54. <value>600000</value>
  55. </property>
  56. <property>
  57. <name>hadoop.kms.current.key.cache.timeout.ms</name>
  58. <value>30000</value>
  59. </property>
  60. ```
  61. $H3 KMS Aggregated Audit logs
  62. Audit logs are aggregated for API accesses to the GET\_KEY\_VERSION, GET\_CURRENT\_KEY, DECRYPT\_EEK, GENERATE\_EEK operations.
  63. Entries are grouped by the (user,key,operation) combined key for a configurable aggregation interval after which the number of accesses to the specified end-point by the user for a given key is flushed to the audit log.
  64. The Aggregation interval is configured via the property :
  65. <property>
  66. <name>hadoop.kms.aggregation.delay.ms</name>
  67. <value>10000</value>
  68. </property>
  69. $H3 Start/Stop the KMS
  70. To start/stop KMS use KMS's bin/kms.sh script. For example:
  71. hadoop-${project.version} $ sbin/kms.sh start
  72. NOTE: Invoking the script without any parameters list all possible parameters (start, stop, run, etc.). The `kms.sh` script is a wrapper for Tomcat's `catalina.sh` script that sets the environment variables and Java System properties required to run KMS.
  73. $H3 Embedded Tomcat Configuration
  74. To configure the embedded Tomcat go to the `share/hadoop/kms/tomcat/conf`.
  75. KMS pre-configures the HTTP and Admin ports in Tomcat's `server.xml` to 16000 and 16001.
  76. Tomcat logs are also preconfigured to go to Hadoop's `logs/` directory.
  77. The following environment variables (which can be set in KMS's `etc/hadoop/kms-env.sh` script) can be used to alter those values:
  78. * KMS_HTTP_PORT
  79. * KMS_ADMIN_PORT
  80. * KMS_MAX_THREADS
  81. * KMS_MAX_HTTP_HEADER_SIZE
  82. * KMS_LOGNOTE: You need to restart the KMS for the configuration changes to take effect.
  83. $H3 Loading native libraries
  84. The following environment variable (which can be set in KMS's `etc/hadoop/kms-env.sh` script) can be used to specify the location of any required native libraries. For eg. Tomact native Apache Portable Runtime (APR) libraries:
  85. * JAVA_LIBRARY_PATH
  86. $H3 KMS Security Configuration
  87. $H4 Enabling Kerberos HTTP SPNEGO Authentication
  88. Configure the Kerberos `etc/krb5.conf` file with the information of your KDC server.
  89. Create a service principal and its keytab for the KMS, it must be an `HTTP` service principal.
  90. Configure KMS `etc/hadoop/kms-site.xml` with the correct security values, for example:
  91. ```xml
  92. <property>
  93. <name>hadoop.kms.authentication.type</name>
  94. <value>kerberos</value>
  95. </property>
  96. <property>
  97. <name>hadoop.kms.authentication.kerberos.keytab</name>
  98. <value>${user.home}/kms.keytab</value>
  99. </property>
  100. <property>
  101. <name>hadoop.kms.authentication.kerberos.principal</name>
  102. <value>HTTP/localhost</value>
  103. </property>
  104. <property>
  105. <name>hadoop.kms.authentication.kerberos.name.rules</name>
  106. <value>DEFAULT</value>
  107. </property>
  108. ```
  109. NOTE: You need to restart the KMS for the configuration changes to take effect.
  110. $H4 KMS Proxyuser Configuration
  111. Each proxyuser must be configured in `etc/hadoop/kms-site.xml` using the following properties:
  112. ```xml
  113. <property>
  114. <name>hadoop.kms.proxyuser.#USER#.users</name>
  115. <value>*</value>
  116. </property>
  117. <property>
  118. <name>hadoop.kms.proxyuser.#USER#.groups</name>
  119. <value>*</value>
  120. </property>
  121. <property>
  122. <name>hadoop.kms.proxyuser.#USER#.hosts</name>
  123. <value>*</value>
  124. </property>
  125. ```
  126. `#USER#` is the username of the proxyuser to configure.
  127. The `users` property indicates the users that can be impersonated.
  128. The `groups` property indicates the groups users being impersonated must belong to.
  129. At least one of the `users` or `groups` properties must be defined. If both are specified, then the configured proxyuser will be able to impersonate and user in the `users` list and any user belonging to one of the groups in the `groups` list.
  130. The `hosts` property indicates from which host the proxyuser can make impersonation requests.
  131. If `users`, `groups` or `hosts` has a `*`, it means there are no restrictions for the proxyuser regarding users, groups or hosts.
  132. $H4 KMS over HTTPS (SSL)
  133. To configure KMS to work over HTTPS the following 2 properties must be set in the `etc/hadoop/kms_env.sh` script (shown with default values):
  134. * KMS_SSL_KEYSTORE_FILE=$HOME/.keystore
  135. * KMS_SSL_KEYSTORE_PASS=password
  136. In the KMS `tomcat/conf` directory, replace the `server.xml` file with the provided `ssl-server.xml` file.
  137. You need to create an SSL certificate for the KMS. As the `kms` Unix user, using the Java `keytool` command to create the SSL certificate:
  138. $ keytool -genkey -alias tomcat -keyalg RSA
  139. You will be asked a series of questions in an interactive prompt. It will create the keystore file, which will be named **.keystore** and located in the `kms` user home directory.
  140. The password you enter for "keystore password" must match the value of the `KMS_SSL_KEYSTORE_PASS` environment variable set in the `kms-env.sh` script in the configuration directory.
  141. The answer to "What is your first and last name?" (i.e. "CN") must be the hostname of the machine where the KMS will be running.
  142. NOTE: You need to restart the KMS for the configuration changes to take effect.
  143. $H4 KMS Access Control
  144. KMS ACLs configuration are defined in the KMS `etc/hadoop/kms-acls.xml` configuration file. This file is hot-reloaded when it changes.
  145. KMS supports both fine grained access control as well as blacklist for kms operations via a set ACL configuration properties.
  146. A user accessing KMS is first checked for inclusion in the Access Control List for the requested operation and then checked for exclusion in the Black list for the operation before access is granted.
  147. ```xml
  148. <configuration>
  149. <property>
  150. <name>hadoop.kms.acl.CREATE</name>
  151. <value>*</value>
  152. <description>
  153. ACL for create-key operations.
  154. If the user is not in the GET ACL, the key material is not returned
  155. as part of the response.
  156. </description>
  157. </property>
  158. <property>
  159. <name>hadoop.kms.blacklist.CREATE</name>
  160. <value>hdfs,foo</value>
  161. <description>
  162. Blacklist for create-key operations.
  163. If the user is in the Blacklist, the key material is not returned
  164. as part of the response.
  165. </description>
  166. </property>
  167. <property>
  168. <name>hadoop.kms.acl.DELETE</name>
  169. <value>*</value>
  170. <description>
  171. ACL for delete-key operations.
  172. </description>
  173. </property>
  174. <property>
  175. <name>hadoop.kms.blacklist.DELETE</name>
  176. <value>hdfs,foo</value>
  177. <description>
  178. Blacklist for delete-key operations.
  179. </description>
  180. </property>
  181. <property>
  182. <name>hadoop.kms.acl.ROLLOVER</name>
  183. <value>*</value>
  184. <description>
  185. ACL for rollover-key operations.
  186. If the user is not in the GET ACL, the key material is not returned
  187. as part of the response.
  188. </description>
  189. </property>
  190. <property>
  191. <name>hadoop.kms.blacklist.ROLLOVER</name>
  192. <value>hdfs,foo</value>
  193. <description>
  194. Blacklist for rollover-key operations.
  195. </description>
  196. </property>
  197. <property>
  198. <name>hadoop.kms.acl.GET</name>
  199. <value>*</value>
  200. <description>
  201. ACL for get-key-version and get-current-key operations.
  202. </description>
  203. </property>
  204. <property>
  205. <name>hadoop.kms.blacklist.GET</name>
  206. <value>hdfs,foo</value>
  207. <description>
  208. ACL for get-key-version and get-current-key operations.
  209. </description>
  210. </property>
  211. <property>
  212. <name>hadoop.kms.acl.GET_KEYS</name>
  213. <value>*</value>
  214. <description>
  215. ACL for get-keys operation.
  216. </description>
  217. </property>
  218. <property>
  219. <name>hadoop.kms.blacklist.GET_KEYS</name>
  220. <value>hdfs,foo</value>
  221. <description>
  222. Blacklist for get-keys operation.
  223. </description>
  224. </property>
  225. <property>
  226. <name>hadoop.kms.acl.GET_METADATA</name>
  227. <value>*</value>
  228. <description>
  229. ACL for get-key-metadata and get-keys-metadata operations.
  230. </description>
  231. </property>
  232. <property>
  233. <name>hadoop.kms.blacklist.GET_METADATA</name>
  234. <value>hdfs,foo</value>
  235. <description>
  236. Blacklist for get-key-metadata and get-keys-metadata operations.
  237. </description>
  238. </property>
  239. <property>
  240. <name>hadoop.kms.acl.SET_KEY_MATERIAL</name>
  241. <value>*</value>
  242. <description>
  243. Complimentary ACL for CREATE and ROLLOVER operation to allow the client
  244. to provide the key material when creating or rolling a key.
  245. </description>
  246. </property>
  247. <property>
  248. <name>hadoop.kms.blacklist.SET_KEY_MATERIAL</name>
  249. <value>hdfs,foo</value>
  250. <description>
  251. Complimentary Blacklist for CREATE and ROLLOVER operation to allow the client
  252. to provide the key material when creating or rolling a key.
  253. </description>
  254. </property>
  255. <property>
  256. <name>hadoop.kms.acl.GENERATE_EEK</name>
  257. <value>*</value>
  258. <description>
  259. ACL for generateEncryptedKey
  260. CryptoExtension operations
  261. </description>
  262. </property>
  263. <property>
  264. <name>hadoop.kms.blacklist.GENERATE_EEK</name>
  265. <value>hdfs,foo</value>
  266. <description>
  267. Blacklist for generateEncryptedKey
  268. CryptoExtension operations
  269. </description>
  270. </property>
  271. <property>
  272. <name>hadoop.kms.acl.DECRYPT_EEK</name>
  273. <value>*</value>
  274. <description>
  275. ACL for decrypt EncryptedKey
  276. CryptoExtension operations
  277. </description>
  278. </property>
  279. <property>
  280. <name>hadoop.kms.blacklist.DECRYPT_EEK</name>
  281. <value>hdfs,foo</value>
  282. <description>
  283. Blacklist for decrypt EncryptedKey
  284. CryptoExtension operations
  285. </description>
  286. </property>
  287. </configuration>
  288. ```
  289. $H4 Key Access Control
  290. KMS supports access control for all non-read operations at the Key level. All Key Access operations are classified as :
  291. * MANAGEMENT - createKey, deleteKey, rolloverNewVersion
  292. * GENERATE_EEK - generateEncryptedKey, warmUpEncryptedKeys
  293. * DECRYPT_EEK - decryptEncryptedKey
  294. * READ - getKeyVersion, getKeyVersions, getMetadata, getKeysMetadata, getCurrentKey
  295. * ALL - all of the above
  296. These can be defined in the KMS `etc/hadoop/kms-acls.xml` as follows
  297. For all keys for which a key access has not been explicitly configured, It is possible to configure a default key access control for a subset of the operation types.
  298. It is also possible to configure a "whitelist" key ACL for a subset of the operation types. The whitelist key ACL is a whitelist in addition to the explicit or default per-key ACL. That is, if no per-key ACL is explicitly set, a user will be granted access if they are present in the default per-key ACL or the whitelist key ACL. If a per-key ACL is explicitly set, a user will be granted access if they are present in the per-key ACL or the whitelist key ACL.
  299. If no ACL is configured for a specific key AND no default ACL is configured AND no root key ACL is configured for the requested operation, then access will be DENIED.
  300. **NOTE:** The default and whitelist key ACL does not support `ALL` operation qualifier.
  301. ```xml
  302. <property>
  303. <name>key.acl.testKey1.MANAGEMENT</name>
  304. <value>*</value>
  305. <description>
  306. ACL for create-key, deleteKey and rolloverNewVersion operations.
  307. </description>
  308. </property>
  309. <property>
  310. <name>key.acl.testKey2.GENERATE_EEK</name>
  311. <value>*</value>
  312. <description>
  313. ACL for generateEncryptedKey operations.
  314. </description>
  315. </property>
  316. <property>
  317. <name>key.acl.testKey3.DECRYPT_EEK</name>
  318. <value>admink3</value>
  319. <description>
  320. ACL for decryptEncryptedKey operations.
  321. </description>
  322. </property>
  323. <property>
  324. <name>key.acl.testKey4.READ</name>
  325. <value>*</value>
  326. <description>
  327. ACL for getKeyVersion, getKeyVersions, getMetadata, getKeysMetadata,
  328. getCurrentKey operations
  329. </description>
  330. </property>
  331. <property>
  332. <name>key.acl.testKey5.ALL</name>
  333. <value>*</value>
  334. <description>
  335. ACL for ALL operations.
  336. </description>
  337. </property>
  338. <property>
  339. <name>whitelist.key.acl.MANAGEMENT</name>
  340. <value>admin1</value>
  341. <description>
  342. whitelist ACL for MANAGEMENT operations for all keys.
  343. </description>
  344. </property>
  345. <!--
  346. 'testKey3' key ACL is defined. Since a 'whitelist'
  347. key is also defined for DECRYPT_EEK, in addition to
  348. admink3, admin1 can also perform DECRYPT_EEK operations
  349. on 'testKey3'
  350. -->
  351. <property>
  352. <name>whitelist.key.acl.DECRYPT_EEK</name>
  353. <value>admin1</value>
  354. <description>
  355. whitelist ACL for DECRYPT_EEK operations for all keys.
  356. </description>
  357. </property>
  358. <property>
  359. <name>default.key.acl.MANAGEMENT</name>
  360. <value>user1,user2</value>
  361. <description>
  362. default ACL for MANAGEMENT operations for all keys that are not
  363. explicitly defined.
  364. </description>
  365. </property>
  366. <property>
  367. <name>default.key.acl.GENERATE_EEK</name>
  368. <value>user1,user2</value>
  369. <description>
  370. default ACL for GENERATE_EEK operations for all keys that are not
  371. explicitly defined.
  372. </description>
  373. </property>
  374. <property>
  375. <name>default.key.acl.DECRYPT_EEK</name>
  376. <value>user1,user2</value>
  377. <description>
  378. default ACL for DECRYPT_EEK operations for all keys that are not
  379. explicitly defined.
  380. </description>
  381. </property>
  382. <property>
  383. <name>default.key.acl.READ</name>
  384. <value>user1,user2</value>
  385. <description>
  386. default ACL for READ operations for all keys that are not
  387. explicitly defined.
  388. </description>
  389. </property>
  390. ```
  391. $H3 KMS Delegation Token Configuration
  392. KMS delegation token secret manager can be configured with the following properties:
  393. ```xml
  394. <property>
  395. <name>hadoop.kms.authentication.delegation-token.update-interval.sec</name>
  396. <value>86400</value>
  397. <description>
  398. How often the master key is rotated, in seconds. Default value 1 day.
  399. </description>
  400. </property>
  401. <property>
  402. <name>hadoop.kms.authentication.delegation-token.max-lifetime.sec</name>
  403. <value>604800</value>
  404. <description>
  405. Maximum lifetime of a delagation token, in seconds. Default value 7 days.
  406. </description>
  407. </property>
  408. <property>
  409. <name>hadoop.kms.authentication.delegation-token.renew-interval.sec</name>
  410. <value>86400</value>
  411. <description>
  412. Renewal interval of a delagation token, in seconds. Default value 1 day.
  413. </description>
  414. </property>
  415. <property>
  416. <name>hadoop.kms.authentication.delegation-token.removal-scan-interval.sec</name>
  417. <value>3600</value>
  418. <description>
  419. Scan interval to remove expired delegation tokens.
  420. </description>
  421. </property>
  422. ```
  423. $H3 Using Multiple Instances of KMS Behind a Load-Balancer or VIP
  424. KMS supports multiple KMS instances behind a load-balancer or VIP for scalability and for HA purposes.
  425. When using multiple KMS instances behind a load-balancer or VIP, requests from the same user may be handled by different KMS instances.
  426. KMS instances behind a load-balancer or VIP must be specially configured to work properly as a single logical service.
  427. $H4 HTTP Kerberos Principals Configuration
  428. When KMS instances are behind a load-balancer or VIP, clients will use the hostname of the VIP. For Kerberos SPNEGO authentication, the hostname of the URL is used to construct the Kerberos service name of the server, `HTTP/#HOSTNAME#`. This means that all KMS instances must have a Kerberos service name with the load-balancer or VIP hostname.
  429. In order to be able to access directly a specific KMS instance, the KMS instance must also have Keberos service name with its own hostname. This is required for monitoring and admin purposes.
  430. Both Kerberos service principal credentials (for the load-balancer/VIP hostname and for the actual KMS instance hostname) must be in the keytab file configured for authentication. And the principal name specified in the configuration must be '\*'. For example:
  431. ```xml
  432. <property>
  433. <name>hadoop.kms.authentication.kerberos.principal</name>
  434. <value>*</value>
  435. </property>
  436. ```
  437. **NOTE:** If using HTTPS, the SSL certificate used by the KMS instance must be configured to support multiple hostnames (see Java 7 `keytool` SAN extension support for details on how to do this).
  438. $H4 HTTP Authentication Signature
  439. KMS uses Hadoop Authentication for HTTP authentication. Hadoop Authentication issues a signed HTTP Cookie once the client has authenticated successfully. This HTTP Cookie has an expiration time, after which it will trigger a new authentication sequence. This is done to avoid triggering the authentication on every HTTP request of a client.
  440. A KMS instance must verify the HTTP Cookie signatures signed by other KMS instances. To do this all KMS instances must share the signing secret.
  441. This secret sharing can be done using a Zookeeper service which is configured in KMS with the following properties in the `kms-site.xml`:
  442. ```xml
  443. <property>
  444. <name>hadoop.kms.authentication.signer.secret.provider</name>
  445. <value>zookeeper</value>
  446. <description>
  447. Indicates how the secret to sign the authentication cookies will be
  448. stored. Options are 'random' (default), 'string' and 'zookeeper'.
  449. If using a setup with multiple KMS instances, 'zookeeper' should be used.
  450. </description>
  451. </property>
  452. <property>
  453. <name>hadoop.kms.authentication.signer.secret.provider.zookeeper.path</name>
  454. <value>/hadoop-kms/hadoop-auth-signature-secret</value>
  455. <description>
  456. The Zookeeper ZNode path where the KMS instances will store and retrieve
  457. the secret from.
  458. </description>
  459. </property>
  460. <property>
  461. <name>hadoop.kms.authentication.signer.secret.provider.zookeeper.connection.string</name>
  462. <value>#HOSTNAME#:#PORT#,...</value>
  463. <description>
  464. The Zookeeper connection string, a list of hostnames and port comma
  465. separated.
  466. </description>
  467. </property>
  468. <property>
  469. <name>hadoop.kms.authentication.signer.secret.provider.zookeeper.auth.type</name>
  470. <value>kerberos</value>
  471. <description>
  472. The Zookeeper authentication type, 'none' or 'sasl' (Kerberos).
  473. </description>
  474. </property>
  475. <property>
  476. <name>hadoop.kms.authentication.signer.secret.provider.zookeeper.kerberos.keytab</name>
  477. <value>/etc/hadoop/conf/kms.keytab</value>
  478. <description>
  479. The absolute path for the Kerberos keytab with the credentials to
  480. connect to Zookeeper.
  481. </description>
  482. </property>
  483. <property>
  484. <name>hadoop.kms.authentication.signer.secret.provider.zookeeper.kerberos.principal</name>
  485. <value>kms/#HOSTNAME#</value>
  486. <description>
  487. The Kerberos service principal used to connect to Zookeeper.
  488. </description>
  489. </property>
  490. ```
  491. $H4 Delegation Tokens
  492. TBD
  493. $H3 KMS HTTP REST API
  494. $H4 Create a Key
  495. *REQUEST:*
  496. POST http://HOST:PORT/kms/v1/keys
  497. Content-Type: application/json
  498. {
  499. "name" : "<key-name>",
  500. "cipher" : "<cipher>",
  501. "length" : <length>, //int
  502. "material" : "<material>", //base64
  503. "description" : "<description>"
  504. }
  505. *RESPONSE:*
  506. 201 CREATED
  507. LOCATION: http://HOST:PORT/kms/v1/key/<key-name>
  508. Content-Type: application/json
  509. {
  510. "name" : "versionName",
  511. "material" : "<material>", //base64, not present without GET ACL
  512. }
  513. $H4 Rollover Key
  514. *REQUEST:*
  515. POST http://HOST:PORT/kms/v1/key/<key-name>
  516. Content-Type: application/json
  517. {
  518. "material" : "<material>",
  519. }
  520. *RESPONSE:*
  521. 200 OK
  522. Content-Type: application/json
  523. {
  524. "name" : "versionName",
  525. "material" : "<material>", //base64, not present without GET ACL
  526. }
  527. $H4 Delete Key
  528. *REQUEST:*
  529. DELETE http://HOST:PORT/kms/v1/key/<key-name>
  530. *RESPONSE:*
  531. 200 OK
  532. $H4 Get Key Metadata
  533. *REQUEST:*
  534. GET http://HOST:PORT/kms/v1/key/<key-name>/_metadata
  535. *RESPONSE:*
  536. 200 OK
  537. Content-Type: application/json
  538. {
  539. "name" : "<key-name>",
  540. "cipher" : "<cipher>",
  541. "length" : <length>, //int
  542. "description" : "<description>",
  543. "created" : <millis-epoc>, //long
  544. "versions" : <versions> //int
  545. }
  546. $H4 Get Current Key
  547. *REQUEST:*
  548. GET http://HOST:PORT/kms/v1/key/<key-name>/_currentversion
  549. *RESPONSE:*
  550. 200 OK
  551. Content-Type: application/json
  552. {
  553. "name" : "versionName",
  554. "material" : "<material>", //base64
  555. }
  556. $H4 Generate Encrypted Key for Current KeyVersion
  557. *REQUEST:*
  558. GET http://HOST:PORT/kms/v1/key/<key-name>/_eek?eek_op=generate&num_keys=<number-of-keys-to-generate>
  559. *RESPONSE:*
  560. 200 OK
  561. Content-Type: application/json
  562. [
  563. {
  564. "versionName" : "encryptionVersionName",
  565. "iv" : "<iv>", //base64
  566. "encryptedKeyVersion" : {
  567. "versionName" : "EEK",
  568. "material" : "<material>", //base64
  569. }
  570. },
  571. {
  572. "versionName" : "encryptionVersionName",
  573. "iv" : "<iv>", //base64
  574. "encryptedKeyVersion" : {
  575. "versionName" : "EEK",
  576. "material" : "<material>", //base64
  577. }
  578. },
  579. ...
  580. ]
  581. $H4 Decrypt Encrypted Key
  582. *REQUEST:*
  583. POST http://HOST:PORT/kms/v1/keyversion/<version-name>/_eek?ee_op=decrypt
  584. Content-Type: application/json
  585. {
  586. "name" : "<key-name>",
  587. "iv" : "<iv>", //base64
  588. "material" : "<material>", //base64
  589. }
  590. *RESPONSE:*
  591. 200 OK
  592. Content-Type: application/json
  593. {
  594. "name" : "EK",
  595. "material" : "<material>", //base64
  596. }
  597. $H4 Get Key Version
  598. *REQUEST:*
  599. GET http://HOST:PORT/kms/v1/keyversion/<version-name>
  600. *RESPONSE:*
  601. 200 OK
  602. Content-Type: application/json
  603. {
  604. "name" : "versionName",
  605. "material" : "<material>", //base64
  606. }
  607. $H4 Get Key Versions
  608. *REQUEST:*
  609. GET http://HOST:PORT/kms/v1/key/<key-name>/_versions
  610. *RESPONSE:*
  611. 200 OK
  612. Content-Type: application/json
  613. [
  614. {
  615. "name" : "versionName",
  616. "material" : "<material>", //base64
  617. },
  618. {
  619. "name" : "versionName",
  620. "material" : "<material>", //base64
  621. },
  622. ...
  623. ]
  624. $H4 Get Key Names
  625. *REQUEST:*
  626. GET http://HOST:PORT/kms/v1/keys/names
  627. *RESPONSE:*
  628. 200 OK
  629. Content-Type: application/json
  630. [
  631. "<key-name>",
  632. "<key-name>",
  633. ...
  634. ]
  635. $H4 Get Keys Metadata
  636. GET http://HOST:PORT/kms/v1/keys/metadata?key=<key-name>&key=<key-name>,...
  637. *RESPONSE:*
  638. 200 OK
  639. Content-Type: application/json
  640. [
  641. {
  642. "name" : "<key-name>",
  643. "cipher" : "<cipher>",
  644. "length" : <length>, //int
  645. "description" : "<description>",
  646. "created" : <millis-epoc>, //long
  647. "versions" : <versions> //int
  648. },
  649. {
  650. "name" : "<key-name>",
  651. "cipher" : "<cipher>",
  652. "length" : <length>, //int
  653. "description" : "<description>",
  654. "created" : <millis-epoc>, //long
  655. "versions" : <versions> //int
  656. },
  657. ...
  658. ]