Browse Source

AMBARI-2174. Eliminate cleartext LDAPS credentials from Ambari props files. (swagle)

git-svn-id: https://svn.apache.org/repos/asf/incubator/ambari/trunk@1492916 13f79535-47bb-0310-9956-ffa450edef68
Siddharth Wagle 12 years ago
parent
commit
5e79481e9d
19 changed files with 1742 additions and 3274 deletions
  1. 0 3231
      CHANGES.txt
  2. 1 1
      ambari-project/pom.xml
  3. 29 0
      ambari-server/pom.xml
  4. 9 1
      ambari-server/sbin/ambari-server
  5. 69 13
      ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
  6. 0 1
      ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
  7. 0 2
      ambari-server/src/main/java/org/apache/ambari/server/security/CertificateManager.java
  8. 144 0
      ambari-server/src/main/java/org/apache/ambari/server/security/encryption/AESEncryptor.java
  9. 198 0
      ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialProvider.java
  10. 29 0
      ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialStoreService.java
  11. 194 0
      ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialStoreServiceImpl.java
  12. 30 0
      ambari-server/src/main/java/org/apache/ambari/server/security/encryption/EncryptionResult.java
  13. 23 0
      ambari-server/src/main/java/org/apache/ambari/server/security/encryption/MasterKeyService.java
  14. 281 0
      ambari-server/src/main/java/org/apache/ambari/server/security/encryption/MasterKeyServiceImpl.java
  15. 470 18
      ambari-server/src/main/python/ambari-server.py
  16. 54 0
      ambari-server/src/test/java/org/apache/ambari/server/security/encryption/AESEncryptorTest.java
  17. 110 0
      ambari-server/src/test/java/org/apache/ambari/server/security/encryption/CredentialStoreServiceTest.java
  18. 92 0
      ambari-server/src/test/java/org/apache/ambari/server/security/encryption/MasterKeyServiceTest.java
  19. 9 7
      ambari-server/src/test/python/TestAmbaryServer.py

+ 0 - 3231
CHANGES.txt

@@ -1,3231 +0,0 @@
-Ambari Change Log
-
-Notes:
- - Committers should be listed using their login and non-committers
-should be listed by their full name.
- - Please keep the file to a max of 80 characters wide.
- - Put latest commits first in each section.
-
-Trunk (unreleased changes):
-
- INCOMPATIBLE CHANGES 
-
- NEW FEATURES
-
- AMBARI-1945. Ambar-client module to be added in the main pom.xml. (Subin via
- mahadev)
-
- AMBARI-2189. Hadoop2 Monitoring: MapReduce2 does not have service 
- page. (srimanth)
-
- AMBARI-2168. Integrate installing and starting Hadoop 2.0 Services.
- (yusaku)
-
- AMBARI-2167. Support for displaying various stacks and having the user select
- which stack to install. (yusaku)
-
- AMBARI-2163. Add smoke tests as part of starting all services in
- security wizard. (jaimin)
-
- AMBARI-2140. Improve Ambari-agent test coverage. (swagle)
-
- AMBARI-2133. Add Nagios alerts for Hadoop 2.0 in Ambari. (swagle)
-
- AMBARI-2123. Allow the user to specify a non-root ssh user in Install Options.
- (yusaku)
-
- AMBARI-2096. Create smoke test for HISTORYSERVER. (swagle)
-
- AMBARI-2094. Create smoke test for NODEMANAGER component as a part of 
- MapReduce V2 Service check. (swagle)
-
- AMBARI-2093. Add Tez as a configurable Service in Hadoop 2.0 stack. (swagle)
-
- AMBARI-2031. AMBARI-2031. Add clover code coverage profile. 
- (Giridharan Kesavan via swagle)
-
- AMBARI-2050. Create smoke test for RESOURCEMANAGER component. (swagle)
-
- AMBARI-2049. Create ambari agent scripts for MAPREDUCEv2_CLIENT. (swagle)
-
- AMBARI-2048. Create ambari agent scripts for historyserver. (swagle)
-
- AMBARI-2046. Create ambari agent scripts for Hadoop 2.0 installation, node 
- manager. (swagle)
-
- AMBARI-2047. Create ambari agent scripts for yarn client. (swagle)
-
- AMBARI-1679. Create ambari agent scripts for Hadoop 2.0 installation, 
- configuration and management. (swagle)
-
- AMBARI-1680. Add Hadoop 2.0 stack definition to Ambari. (swagle)
-
- AMBARI-1908. HDFS Mirroring: Add Bread Crumbs and Validation. (Arun Kandregula
- via yusaku)
-
- AMBARI-1558. Script to add host components to existing hosts.
- (yusaku via jaimi)
-
- AMBARI-1936. Support for installing on mixed OS versions install + mgmt.
- (swagle)
-
- AMBARI-1924. Allow for users to customize Ganglia gmetad + gmond user 
- accounts. (Sumit Mohanty via swagle)
-
- AMBARI-1923. Allow for users to customize Nagios user accounts. 
- (Sumit Mohanty via swagle)
-
- AMBARI-1922. Support not root ssh via a user that can sudo in as root. 
- (Sumit Mohanty via swagle)
- 
- AMBARI-1914. Add Nagios alerts for Hue service. (swagle)
-
- AMBARI-1895. Refactor ajax requests. (srimanth)
-
- AMBARI-1868. Include stack version as a parameter in manifest. (swagle)
-
- AMBARI-1847. Make single PUT call for multiple host overrides. (srimanth)
-
- AMBARI-1857. Capacity Scheduler: field order for Add/Edit popup. (yusaku via
- srimanth)
-
- AMBARI-1855. Capacity Scheduler: when adding a new queue, populate 
- fields. (yusaku via srimanth)
- 
- AMBARI-1850. Update unit tests. (yusaku via srimanth)
-
- AMBARI-1829. HDFS Mirroring: Display Status and handle maintenance operations 
- like Stop, Suspend, Activate etc. (Arun Kandregula via srimanth)
-
- AMBARI-1840. For global properties show restart for appropriate services
- only. (srimanth)
-
- AMBARI-1800. Add "Admin > Misc" section to Ambari Web to show service user
- accounts. (yusaku)
-
- AMBARI-1756. Add ability to install and edit HUE as a service. (srimanth via 
- yusaku)
-
- AMBARI-1742. HDFS Mirroring: Edit/Delete Cluster. (Arun Kandregula via yusaku)
-
- AMBARI-1723. HDFS Mirroring: Edit/Delete Data Set. (srimanth via yusaku)
-
- AMBARI-1717. Add ability to start and stop all services from Services
- page. (Xi Wang via yusaku)
-
- AMBARI-1716. HDFS Mirroring: Add a cluster. (Arun Kandregula via yusaku)
-
- AMBARI-1710. HDFS Mirroring: Edit/Delete Data Set. (srimanth via yusaku)
-
- AMBARI-1699. HDFS Mirroring: Side Panel of individual jobs page.
- (yusaku)
-
- AMBARI-1698. Host Detail page needs to allow upgrade for host components
- that failed to upgrade. (yusaku)
-
- AMBARI-1696. Capacity Scheduler configuration UI. (yusaku)
-
- AMBARI-1693. HDFS Mirroring: Display Jobs table. (yusaku)
-
- AMBARI-1691. Add filtering by host-level status on Step 9 of Installer.
- (Xi Wang via yusaku)
-
- AMBARI-1668. HDFS Mirroring: Add Data Set Popup. (Arun Kandregula via
- yusaku)
-
- AMBARI-1650. Add Oracle and MySQL option for Oozie during Ambari cluster
- install. (Xi Wang via yusaku)
-
- AMBARI-1610. Expose ability to customize Hive Metastore log dir.
- (yusaku)
-
- AMBARI-1729. Creating smoke test for Hue service. (swagle)
-
- AMBARI-1776. ZooKeeper Servers needs to store correct kerberos principal 
- in zookeeper_jaas.conf. (swagle)
-
- AMBARI-1424. Upgrade enhancements for Ambari 1.3.0. (smohanty)
-
- AMBARI-1763. Integrate Frontend security work to enable security on
- HBase and ZooKeeper. (jaimin)
-
- AMBARI-1754. Add support to ensure that Ambari Server/Agent/Store are all of 
- compatible version. (smohanty)
-
- AMBARI-1752. Backend support for MySQL and Oracle for Oozie and Hive. (swagle)
-
- AMBARI-1751. Ambari oracle-linux as a supported OS type. (swagle)
-
- AMBARI-1728. Cleanup INFO Logging at the ambari agent to make it more useful 
- and less verbose. (swagle)
-
- AMBARI-1676. Ambari upgrade to 1.3.0 (core support). (smohanty)
-
- AMBARI-1708. Remove all hardcoded ports from agent scripts to read from 
- configs. (swagle)
-
- AMBARI-1692. Make changes to agent scripts to support secure HBase and Zk. (swagle)
-
- AMBARI-1707. Upgrade should check if another upgrade request is active as well as 
- if any MASTER components have not stopped. (Sumit Mohanty via swagle)
-
- AMBARI-1673. Configuring Hue to work with a secure HDP cluster and making changes 
- to the Enable Security feature. (swagle)
-
- AMBARI-1663. Allow adding host components to existing hosts. (Xi Wang via
- yusaku)
-
- AMBARI-1653. HDFS Mirroring: Display DataSets table. (yusaku)
-
- AMBARI-1658. Implement API/Service Provider for HDFS mirroring. (tbeerbower)
-
- AMBARI-1704. Add ability for host components to provide their current actual configs. (ncole)
-
- AMBARI-1422. Allow client to specify a "context" value for asynchronous requests (jspeidel)
-
- AMBARI-1599. Add ability to report actual configuration applied to a host. (ncole)
-
- AMBARI-1647. Integrate server and agent changes for upgrade on cluster. 
- (Sumit Mohanty via swagle)
-
- AMBARI-1626. API support to upgrade host component. (Sumit Mohanty via swagle)
-
- AMBARI-1601. Server level action support. (Sumit Mohanty via swagle)
-
- AMBARI-1620. Add heatmaps for Host and Hbase section. (jaimin)
- 
- AMBARI-1634. Integrate Frontend Security work to enable security on
- Oozie, Hive, and WebHCat Server. (jaimin)
-
- AMBARI-1633. Reassign Master Wizard - Step 5. (yusaku)
-
- AMBARI-1585. Creating the agent scripts for Hue server installation and 
- configuration on the Hue host. (swagle)
-
- AMBARI-1618. HDFS Mirroring: Create Mapper, Model, Mock Data for Cluster.
- (Arun Kandregula via yusaku)
-
- AMBARI-1607. HDFS Mirroring: Create Mapper, Model and Mock Data.
- (Arun Kandregula via yusaku)
-
- AMBARI-1602. Edit User - drop the requirement to specify the old 
- password. (swagle)
-
- AMBARI-1406. Provide API support for including query string in http message body. (jspeidel)
-
- AMBARI-1592. Change how configurations are propagated (ncole)
-
- AMBARI-1593. Change host override JSON to include version tag (ncole)
-
- AMBARI-1545. Integrate Frontend Security work to enable security on HDFS
-
- AMBARI-1555. Upgrade should validate that the from->to version is an allowed 
- combination. (Sumit Mohanty via swagle)
-
- AMBARI-1568. Update the version of ambari artifacts to 1.3.0 snapshot (ncole)
-
- AMBARI-1563. API Support:  Host-component resource should include its current 
- HA active/passive status. (Sumit Mohanty via swagle)
-
- AMBARI-1560. Upgrade action/task support in server. (Sumit Mohanty via swagle)
-
- AMBARI-1553. List cluster-level configurations with host-level, if any (ncole)
-
- AMBARI-1557. Adding Hue service to the HDP stack definition along with the 
- necessary configuration properties. (swagle)
-
- AMBARI-1554. API support for current version of Stack and available versions 
- to upgrade (Sumit Mohanty via swagle) 
-
- AMBARI-1511. Add ability to override configurations at the host level (ncole)
-
- AMBARI-1550. Modify existing puppet manifests to allow installing/configuring 
- multiple masters. (swagle)
-
- AMBARI-1545. Integrate Frontend Security work to enable security on HDFS
- and MapReduce installed cluster. (jaimin)
-
- AMBARI-1528. Upgrade request support at Ambari. (Sumit Mohanty via swagle)
-
- AMBARI-1541. Upgrade task support in agent. (Sumit Mohanty via swagle)
-
- AMBARI-1540. Reassign Master Wizard - Steps 3 and 4 (reconfigure
- component and review). (yusaku)
-
- AMBARI-1538. Stack Upgrade Wizard - Step 3 (Progress and Retry). (yusaku) 
-
- AMBARI-1509. Frontend: For service configurations provide ability to 
- enter host level exceptions (srimanth)
-
- AMBARI-1508. Introduce a new section "Capacity scheduler" under MapReduce
- Service in step7 (Configure services) of Installer and Service 
- Reconfig. (jaimin)
-
- AMBARI-1490. Implement initial layout for "Add security" wizard. (jaimin)
-
- AMBARI-1483. Reassign Master Wizard - Step 2. (yusaku)
-
- AMBARI-1482. Reassign Master Wizard - Step 1. (yusaku)
-
- AMBARI-1481. Stack Upgrade Wizard - Step 2 (confirm and check all master
- components are running). (yusaku)
-
- AMBARI-1469. Allow user to add multiple HBase masters in Install Wizard.
- (yusaku)
-
- AMBARI-1468. Stack Upgrade Wizard - Step 1 (show services and versions).
- (yusaku)
-
- AMBARI-1459. Add Admin > Cluster page. (yusaku)
-
- AMBARI-1454. Service page: add "Reassign <Master Component>" action items
- to "Maintenance" pulldown. (yusaku)
-
- AMBARI-1349. Expose host-specific Nagios alerts in Ambari Web. (yusaku)
-
- AMBARI-1294. Add isEmpty() query operator support. (jspeidel)
-
- AMBARI-1280. Support explicit predicate grouping in API queries. (jspeidel)
-
- AMBARI-1180. Display host check status results given by the agent as part
- of host registration. (yusaku)
-
- AMBARI-1252. Fetch Nagios alerts through Ambari Server and not directly
- from Nagios Server. (srimanth via yusaku)
-
- AMBARI-1237. Expose Nagios alerts via Rest API. (Nate Cole via jspeidel)
-
- AMBARI-1163. During agent registration and heartbeat, send information about
- various hadoop artifacts back to Ambari. (Nate Cole via mahadev)
-
- AMBARI-1194. API support for cascade delete of a specified cluster
- (Tom Beerbower via mahadev)
-
- AMBARI-1255. Make the agent hostname determination scriptable. 
- (mahadev)
-
- AMBARI-1267. Store example Hive Queries somewhere in Ambari that's easily
- accessible for demo/test purposes. (mahadev)
-
- AMBARI-1447. Report current Stack version for all host components.
- (Sumit Mohanty via mahadev)
-
- IMPROVEMENTS
-
- AMBARI-2234. Host Details Page: Update host component actions and icons
- depending on state. (yusaku)
-
- AMBARI-2199. Add a legend to Job Timeline. (billie via yusaku)
-
- AMBARI-2212. Change config loading mechanism to allow for different
- stack versions. (yusaku)
-
- AMBARI-2207. Add unit tests for Utils. (yusaku)
-
- AMBARI-2206. Add unit tests for wizard steps 2, 5, and 6. (yusaku)
-
- AMBARI-2194. Hadoop2 Installer: MapReduce2 shows only advanced section. 
- (srimanth)
-
- AMBARI-2187. Hadoop2 Monitoring: Jobs page should be hidden when HDP 2.0.x 
- stack is installed. (srimanth)
-
- AMBARI-2188. Update mock json data for Test mode. (srimanth) 
-
- AMBARI-2169. Going from Hosts page to Host Details page and back should
- preserve the filters, sort order, and pagination. (yusaku)
-
- AMBARI-2180. Remove "0.1" stack definition since its never been used and is
- redundant. (mahadev)
-
- AMBARI-2158. Certain queries with metrics takes a long time (3 minutes on
- a 450-node cluster). (tbeerbower)
-
- AMBARI-2150. Reassign Master Wizard: start the service after reassigning the
- master. (yusaku)
-
- AMBARI-1887. Heatmap should display full details on mapped metric.
- (Sandeep Baldawa via yusaku)
-
- AMBARI-2137. Enable WebHDFS by default. (yusaku)
-
- AMBARI-2122. Improve navigation when jumping from one page to Host Details
- page and back. (yusaku)
-
- AMBARI-2121. Allow Nagios user to be customized during install. (yusaku)
-
- AMBARI-2111. Enable customization of smoke test user. (yusaku)
-
- AMBARI-2191. Report status for YARN and MAPREDUCE2. (ncole)
-
- AMBARI-2147. Added config change auditing. (ncole)
-
- AMBARI-2110. Update hive-site.xml, set fs.file.impl.disable.cache=true.
- (mahadev)
-
- AMBARI-2070. Changing service directories should popup a confirmation/warning
- dialog upon save. (yusaku)
-
- AMBARI-2061. HBase Heatmaps: clean up labels and units. (yusaku)
-
- AMBARI-2042. Update Ambari logo. (yusaku)
-
- AMBARI-2040. Customize Services page: reduce padding to prevent tabs from
- wrapping. (yusaku)
-
- AMBARI-1388. Document management API. (tbeerbower)
-
- AMBARI-2030. Make frontend changes to account for the host component status
- UNKNOWN. (yusaku)
-
- AMBARI-2028. Customize Services: make the services display consistent.
- (yusaku)
-
- AMBARI-2027. Add validation checks for Add Property on custom site configs.
- (yusaku)
-
- AMBARI-2017. Admin Misc page tweaks. (yusaku)
-
- AMBARI-2022. Service Component metric collection API takes over a minute
- on large cluster. (tbeerbower)
-
- AMBARI-2005. When adding a component to a host (after cluster deployment),
- UI should warn that nagios server need to be restarted. (yusaku)
-
- AMBARI-2004. Background Operation Popup needs label/styling fixes.
- (yusaku)
-
- AMBARI-2003. Hosts tab: clicking on red badge should not toggle "Alerts"
- filter. (yusaku)
-
- AMBARI-2002. Dashboard: remove "Version" row from HDFS, MapReduce, and
- HBase dashboard summary. (yusaku)
-
- AMBARI-1982. Disallow editing Hadoop log/PID directories post install.
- (yusaku)
-
- AMBARI-1981. Expose ability to customize the username for running
- Ganglia daemons. (yusaku)
-
- AMBARI-1987. Add unit tests for admin/cluster page and cluster loading.
- (yusaku)
-
- AMBARI-1929. Make the default stack and version configurable via mvn build.
- (yusaku)
-
- AMBARI-1967. Add smoke test user customization toggle via App.supports.
- (yusaku)
-
- AMBARI-1964. Add ability to toggle on/off assigning masters for new hosts
- being added via Add Host wizard. (yusaku)
-
- AMBARI-1958. Update alert icon legend on Hosts page. (yusaku)
-
- AMBARI-1955. Add ability to turn on/off HUE support. (yusaku)
-
- AMBARI-1941. Create a flag to toggle the support for multiple
- HBase Masters. (jaimin)
-
- AMBARI-1940. Create a flag to toggle the support for additional database 
- support in Hive and Oozie. (jaimin)
-
- AMBARI-1928. Make experimental Ambari Web features toggleable
- via mvn build. (yusaku via jaimin)
- 
- AMBARI-1906. Make experimental Ambari Web features toggleable 
- (turned on/off via config file). (yusaku via jaimin)
-
- AMBARI-1905. Test mode is broken. (yusaku via jaimin)
-
- AMBARI-1904. Update default stack version to 1.3.0. (yusaku via jaimin)
-
- AMBARI-1921. Change tasks order in popup on deploy page. (jaimin)
-
- AMBARI-1913. Security Wizard - "Kerberos security is disabled on the cluster"
- should not be shown in red. (jaimin)
-
- AMBARI-1403. Remove SPI dependencies on other code. (tbeerbower)
-
- AMBARI-1892. Restrict user on a Disable security popup while the poccessi
- is in progress. (jaimin)
-
- AMBARI-1879. Show error message when hostname is undefined for quick_links.
- (yusaku)
-
- AMBARI-1877. Reassign Master Wizard, Step 2: prevent proceed next without
- changing target host. (yusaku)
-
- AMBARI-1876. Capacity Scheduler: implement user/group and admin user/group
- validation rules. (yusaku)
-
- AMBARI-1864. Remove package dependency on mocha-phantomjs. (yusaku)
-
- AMBARI-1844. Need ability to update multiple hosts in 1 PUT call.
- (tbeerbower)
-
- AMBARI-1845. Server log is being flooded with log messages.
- (tbeerbower)
-
- AMBARI-1825. Minor label change for the buttons to start/stop all services
- in the side nav. (yusaku)
-
- AMBARI-1824. Background operations popup: progress bar styling. (yusaku)
-
- AMBARI-1823. Add hover tooltip for the green host health status icon in
- Hosts page and Host Detail page. (yusaku)
-
- AMBARI-1815. After modifying custom configs, properties are shown as blank,
- leading to corrupt core-site.xml upon save. (yusaku)
-
- AMBARI-1814. Refactor configurations functionality. (yusaku)
-
- AMBARI-1809. Specify request context for Smoke test. (yusaku)
-
- AMBARI-1808. Add ability to customize the smoke test user (frontend changes).
- (yusaku)
-
- AMBARI-1807. Disallow $ from usernames created via Misc in Install Wizard.
- (yusaku)
-
- AMBARI-1805. Minor text change in the Assign Masters page. (yusaku)
-
- AMBARI-1798. Explicitly state that FQDNs need to be used when specifying
- hosts to add to the cluster. (yusaku)
-
- AMBARI-1785. Provide restart service indicator to all services. (srimanth
- via yusaku)
-
- AMBARI-1773. Integrate MySQL/Oracle options for Hive and Oozie. (Xi Wang via
- yusaku)
-
- AMBARI-1758. Reassign Master Wizard: Integration and hardening. 
- (srimanth via yusaku)
-
- AMBARI-1743. Stack Upgade Wizard - integrate with Ambari 1.3.0 repo.
- (srimanth via yusaku)
-
- AMBARI-1741. Exceptions thrown when using host overrides during install.
- (srimanth via yusaku)
-
- AMBARI-1740. Provide restart message in service config page. (srimanth via
- yusaku)
-
- AMBARI-1722. Frontend support for LDAP Group to Ambari Role Mapping.
- (srimanth via yusaku)
-
- AMBARI-1718. If there is no HBase master in active status, show proper
- error message. (jaimin via yusaku)
-
- AMBARI-1701. Stack Upgrade Wizard: Integrate with API. (yusaku)
- 
- AMBARI-1700. HDFS Mirroring: Display Jobs table (refactor). (yusaku)
-
- AMBARI-1697. UI changes on HBase service page for HBase Multi-master/HA.
- (yusaku)
-
- AMBARI-1782. Security wizard navigation: Restrict user on step3 until
- the decision of the step is reached. (jaimin)
-
- AMBARI-1652. Background operation display enhancements. (yusaku)
-
- AMBARI-1686. Implement Test IvoryService to functional test mirroring API.
- (tbeerbower)
-
- AMBARI-1672. Security Wizard - integrate with cluster-level config API. (jaimin)
-
- AMBARI-1669. Security Wizard UI tweaks. (jaimin)
-
- AMBARI-1630. Expose HUE config parameters through Ambari Web. (yusaku)
-
- AMBARI-1595. Add Oracle option for Hive Metastore during Ambari
- cluster install. (Xi Wang via yusaku)
-
- AMBARI-1632. Provide property filtering capability on service config
- sections. (yusaku)
-
- AMBARI-1631. Security Wizard - integrate host progress popup. (yusaku)
-
- AMBARI-1604. Refactor wizard classes. (yusaku)
-
- AMBARI-1583. Add unit tests for various Ambari Web components. (yusaku)
-
- AMBARI-1491. Add task plots to job swimlane diagram. (billie via yusaku)
-
- AMBARI-1584. Stack Upgrade Wizard - integrate host progress popup.
- (yusaku)
-
- AMBARI-1581. Host progress popup - generic component for showing progress
- on async operations. (yusaku)
-
- AMBARI-1542. Provide remove, restore default, and cancel actions for
- service config properties. (srimanth via yusaku)
-
- AMBARI-1580. Stack Upgrade Wizard - resume upon page refresh / login.
- (yusaku)
-
- AMBARI-1578. Add host wizard - support assignment of "ZooKeeper Server"
- and "HBase Master". (yusaku)
-
- AMBARI-1546. Improve Cluster Management loading screen. (Xi Wang via yusaku)
-
- AMBARI-1890. Add configuration to test_api.sh script (ncole)
-
- AMBARI-2026. Add WebHCat to live status checks. (ncole)
-
- AMBARI-1975. Add Clover coverage (ncole)
-
- AMBARI-1537. Constrain the width of all wizard popups. (Xi Wang via yusaku)
-
- AMBARI-1536. Hosts page layout fixes. (Xi Wang via yusaku)
-
- AMBARI-1502. Add the ability to assign configuration to a cluster. (Nate Cole via tbeerbower)
-
- AMBARI-1505. Hosts page: add filtering by host status. (yusaku)
-
- AMBARI-1496. Make all service properties reconfigurable. (jaimin)
-
- AMBARI-1477. Improve performance for App.statusMapper. (yusaku)
-
- AMBARI-1484. Reintroduce client-side paging for Hosts table. (yusaku)
-
- AMBARI-1473. Further optimization of querying host information from the
- server. (yusaku)
-
- AMBARI-1472. Update HBase service/host health status. (yusaku)
-
- AMBARI-1471. Refactor ajax calls. (yusaku)
-
- AMBARI-1461. Optimize query for getting service and host component status back
- from the server. (yusaku)
-
- AMBARI-1460. Optimize query call for retrieving host information. (yusaku)
-
- AMBARI-1470. Refactor confirmation popups. (yusaku)
-
- AMBARI-1414. Expose fine grained HDFS capacity metrics in API. (tbeerbower)
-
- AMBARI-1444. Make install, service & host configuration pages reuse same
- configuration code. (srimanth via yusaku)
-
- AMBARI-1457. Improve Job Diagnostics. (Billie Rinaldi via yusaku)
- 
- AMBARI-1453. Move Ambari Web application config from initialize.js to
- another config file. (yusaku)
-
- AMBARI-1450. Remove hard-coded stack version. (yusaku)
-
- AMABRI-1458. Externalize strings from views to messages.js. (yusaku)
-
- AMBARI-1437. Update stack version. (yusaku)
-
- AMBARI-1429. Update API docs. (jspeidel)
-
- AMBARI-1430. Increase UI timeout for long running API operations. (yusaku)
-
- AMBARI-1427. Add ability to increase the time range for the zoomed-in graphs
- beyond last one hour. (yusaku) 
-
- AMBARI-1375. Remove text from templates (main). (jaimin)
-
- AMBARI-1374. Add filter by alerts on the Hosts page. (jaimin)
-
- AMBARI-1373. Since there is the ability to log in to Ambari Web as 
- different users the current user should be indicated. (jaimin)
-
- AMBARI-1366. Nagios alert tweaks. (jaimin)
-
- AMBARI-1365. Make Hosts table update dynamically. (jaimin)
-
- AMBARI-1361. Install progress dialog WARN icon + color. (jaimin)
-
- AMBARI-1347. Expose host-level alerts via nagios_alerts.php with associated
- service component names. (yusaku)
-   
- AMBARI-1348. Externalize strings to messages.js. (yusaku)
-
- AMBARI-1342. Hive client is not installed on Nagios server host.
- (jaimin)
-
- AMBARI-1341. Add Hosts: update the API call for new operator precedence.
- (yusaku) 
-
- AMBARI-1340. Enhance Install/Start/Test progress display. (yusaku) 
-
- AMBARI-1339. Validate usernames in Misc section of Customize Services step
- in Install Wizard. (yusaku)
-
- AMBARI-1335. Show validation error when the user specifies target hosts that
- are already part of the cluster. (yusaku)
-
- AMBARI-1337. Refactor Job Browser filter. (yusaku)
-
- AMBARI-1336. Externalize text to messages.js. (yusaku)
-
- AMBARI-1334. Show hosts that have failed install tasks as "red" to allow the
- user to easily identify source of failure. (yusaku)
-
- AMBARI-1333. Add username validation for Ambari local users. (yusaku) 
-
- AMBARI-1329. Adjust job browser column sizing. (yusaku)
- 
- AMBARI-1327. Add Hosts. Remove existig hosts display. (Alexandr Antonenko via jspeidel)
-
- AMBARI-1326. Remake clearFilters function in app_view (part 3). (srimanth)
- 
- AMBARI-1305. Make sure that Ambari Web renders all elements correctly when
- the browser width is 1024px or narrower (refactor). (Arun Kandregula via 
- yusaku) 
-
- AMBARI-1312. Remake clearFilters function in app_view (part2). (Arun Kandregula
- via yusaku) 
- 
- AMBARI-1309. Remove all text from Apps views, controllers, templates to 
- messages.js. (Arun Kandregula via yusaku)
-
- AMBARI-1308. Properly display Apps page aggregate summary and data table when
- there are no data to be show. (Arun Kandregula via yusaku)
-
- AMBARI-1306. Change color of rack_local_map to #66B366. (yusaku)
-
- AMBARI-1311. Host health indicator should have a tooltip showing few details - 
- refactoring. (Arun Kandregula via yusaku)
-
- AMBARI-1303. Remake clearFilters function in app_view. (Arun Kandregula via
- yusaku)
-
- AMBARI-1302. Minor label cleanup on Jobs Charts popup. (Arun Kandregula via
- yusaku)
-
- AMBARI-1296. Task log popup footer should be statically placed only the
- content should scroll vertically. (Jaimin Jetly via yusaku)
-
- AMBARI-1295. Move cluster name display from the main nav to the top nav.
- (Jaimin Jetly via yusaku)
-
- AMBARI-1268. Improve DAG UI. (billie via yusaku)
-
- AMBARI-1289. App page: remove old ode and fix test mode. (srimanth via
- yusaku)
-
- AMBARI-1279. Make sure that Ambari Web renders all elements correctly when
- the browser width is 1024px or narrower. (srimanth via yusaku)
-
- AMBARI-1274. Shrink top nav height. (srimanth)
- 
- AMBARI-1272. Controller javascripts need comments. (srimanth)
- 
- AMBARI-1271. On Confirm Hosts page, add a link to show the Host Checks popup 
- in the success message. (yusaku via srimanth)
-
- AMBARI-1193. If Install fails, allow user to go back to any previous step so 
- that the user can retry install with different configuration parameters.
- (yusaku via srimanth)
-
- AMBARI-1265. Job Browser - Filter by Input, output and duration. (yusaku)
-
- AMBARI-1263. Refactoring of User Management code. (yusaku)
-
- AMBARI-1254. Modify App Browser to use server-side paging/sorting/filtering.
- (yusaku)
-
- AMBARI-1258. Minor refactoring of User Management code. (yusaku)
-
- AMBARI-1253. Use ember-precompiler-brunch npm plugin. (yusaku)
-
- AMBARI-1236. Display a progress bar during deploy prep. (yusaku)
-
- AMBARI-1249. Update mock data to make App.testMode work. (yusaku)
-
- AMBARI-1239. Host health status should show orange when there is at least one
- slave component on the host with state!=STARTED. (yusaku)
-
- AMBARI-1248. Refactoring of update, services and hosts mapper. (yusaku)
-
- AMBARI-1247. Disable links for previous steps in left nav on Summary step.
- (yusaku)
-
- AMBARI-1246. Add user minor improvements. (yusaku)
-
- AMBARI-1245. Do not let the user go back to the previous step while host
- bootstrap is in progress. (yusaku)
-
- AMBARI-1244. Install Options - line up the Target Hosts section with the rest
- of the page. (yusaku)
-
- AMBARI-1235. Host health indicator should have a tooltip showing details.
- (yusaku)
- 
- AMBARI-1234. On Heatmap host hover, including list of components running.
- (yusaku)
-
- AMBARI-1229. Dashboard - make disk usage pie chart in HDFS summary easier
- to understand. (yusaku)
-
- AMBARI-1228. During Install, show "warn" on hosts that have tasks cancelled.
- (yusaku)
-
- AMBARI-1225. Add Hosts wizard popup is too small. (yusaku)
-
- AMBARI-1224. Drop the "all" option from Hosts > Component Filter and
- Jobs > Users Filter. (yusaku)
-
- AMBARI-1223. Confirm Hosts page: It looks like hosts disappear if you are
- on "Fail" filter and click on "Retry Failed" button. (yusaku)
-
- AMBARI-1222. DAG, Jobs Timeline, and Tasks graphs UI cleanup. (yusaku)
-
- AMBARI-1221. There is no default sort order on Hosts table and the order
- changes on every page refresh - should sort by hostname. (yusaku)
-
- AMBARI-1220. Oozie service summary update. (yusaku)
-
- AMBARI-1218. Refactor Job Browser User filter. (yusaku)
-
- AMBARI-1217. Tighten up spacing for the rows in the Hosts table. (yusaku)
-
- AMBARI-1216. Add filters module. (yusaku)
-
- AMBARI-1215. Refactor hostComponent isSlaves and isMaster and add update
- methods for server mapper. (yusaku)
-
- AMBARI-1214. In any starts fails, "warn" the host and the overall install.
- (yusaku)
-
- AMBARI-1204. Install Wizard: Re-enable configuration of user/group names for
- master component daemons. (yusaku)
-
- AMBARI-1197. Refactor code for graphs. (yusaku)
-
- AMBARI-1196. Automatically update host-level popup info/logs. (yusaku)
-
- AMBARI-1189. Add App.Job class. (yusaku)
-
- AMBARI-1188. Refactor isClient computed property for HostComponent class.
- (yusaku)
-
- AMBARI-1186. Add Run class to represent a job run. (yusaku)
-
- AMBARI-1185. Refactor the method to check if the user is an admin.
- (yusaku)
-
- AMBARI-1183. Directories in the service config textarea should not wrap.
- (yusaku)
-
- AMBARI-1182. Clean up table header UI for sorting and filter clear "x" for
- Jobs table. (yusaku)
-
- AMBARI-1181. Clean up table header UI for sorting and filter clear "x" for
- Hosts table. (yusaku)
-
- AMBARI-1198. Ambari API Performance: Parsing of Ganglia json data is slow.
- (jspeidel via mahadev)
-
- AMBARI-1213. Cleanup python test cases and introduce third party library for
- mock testing python code. (mahadev)
-
- AMBARI-1206. Expose missing metrics on host components. (tbeerbower via
- mahadev)
-
- AMBARI-1205. Cannot persist service configuration when service is started
- (Siddharth Wagle via mahadev)
-
- AMBARI-1262. Apache Ambari point to dev url, need fix in pom.xml. 
- (mahadev)
-
- AMBARI-1207. Remove /hdp as the httpd conf for any of the nagios urls -
- should replace it with ambarinagios or something else.
- (mahadev)
-
- AMBARI-1277. Failing build due to url moved on Suse. (mahadev)
-
- AMBARI-1288. Change "authorization" to "authentication" in props setup for
- LDAP. (mahadev)
-
- AMBARI-1269. Refactor ResourceProvider SPI. (tbeerbower)
- 
- AMBARI-1270. Add predicate objects for checking empty resource category.
- (tbeerbower)
-
- AMBARI-1286. Set version number property in gsInstaller cluster resource
- provider. (tbeerbower)
-
- AMBARI-1287. Monitor for component/service state for gsInstaller resource provider. (tbeerbower)
-
- AMBARI-1260. Remove hard coded JMX port mappings. (Siddharth Wagle via
- mahadev)
-
- AMBARI-1411. Missing unit test coverage for resource providers. (tbeerbower)
-
- AMBARI-1433. Allow capacity scheduler to be configurable via the API's.
- (mahadev)
-
- AMBARI-1435. L2 Cache does not work due to Eclipse Link exception.
- (Sid Wagle via mahadev)
-
- AMBARI-1436. Threads blocking on ClustersImpl.getHost for several minutes.
- (Sid Wagle via mahadev)
-
- AMBARI-1438. Add new stack definition for new stacks. (mahadev)
-
- AMBARI-1448. Enabling stack upgrade via Ambari Server. (mahadev)
-
- AMBARI-1439. rrd file location should be read from global config. (Siddharth
- Wagle via mahadev).
-
- AMBARI-1357. Smoke Tests failing on secure cluster. (Siddharth Wagle via
- mahadev)
-
- AMBARI-1343. Service Check fails after secure install due to wrong kinit
- path. (Siddharth Wagle via mahadev) 
- 
- AMBARI-1465. Minimize Read and Write locks for createHosts. (Siddharth Wagle
- via mahadev)
-
- AMBARI-1466. Optimize ganglia rrd script to be able to respond within
- reasonable time to queries made by the UI. (mahadev)
-
- AMBARI-1474. Upgrade stack definition for HBase for 1.2.2 since the version
- is upgraded. (mahadev)
-
- AMBARI-1475. Update the version of ambari artifacts to 1.2.2 snapshot.
- (mahadev)
-
- AMBARI-1489. Add hadoop-lzo to be one of the rpms to check for before
- installation. (mahadev)
-
- AMBARI-1642. Add ability for maintainence mode in Host Role Component in
- Ambari. (mahadev)
-
- AMBARI-1797. For global site properties, need property to services effected
- map. (mahadev)
-
- AMBARI-1384. WorkflowJsonService service doesn't use the API framework and is
- inconsistent with other API's. (billie)
-
- AMBARI-1871. ambari-agent RPM does not claim ownership of
- /var/lib/ambari-agent. (Matthew Farrellee via mahadev)
-
- AMBARI-1870. ambari-agent RPM claims ownership of /usr/sbin. (Matthew
- Farrellee via mahadev)
-
- BUG FIXES
- 
- AMBARI-2369. B6 setup stops on Postgresql start on SLES11. (swagle) 
-
- AMBARI-2336. Datanode start failed (with Oracle DB) due to RCA setup. 
- (swagle)
-
- AMBARI-2266. Clients are not reconfigured on starting a service after each 
- of the service components are stopped individually. (swagle)
-
- AMBARI-2230. Nagios user is presented in Admin > Misc page when Nagios
- service was not installed. (yusaku)
-
- AMBARI-2231. Service web UI links map to hostnames not resolvable by the
- client. (yusaku)
-
- AMBARI-2239. secure cluster: Namenode and SNamenode should use same
- keytab. (jaimin)
-
- AMBARI-2235. NN enters safe mode upon HDFS restart. (smohanty)
-
- AMBARI-2225. Security fixes with HBase service check. (jaimin)
-
- AMBARI-2233. Ensure version values are used appropriately throughout
- Ambari. (smohanty)
-
- AMBARI-2229. Oracle driver loading broken on agent side. (smohanty)
-
- AMBARI-2228. Fix MySQL and Oracle DDL scripts according to last DB
- changes. (smohanty)
-
- AMBARI-2198. Avoid using $::fqdn in puppet which uses the fqdn value from
- puppet/facter, instead pass in the hostname from
- python socket.getfqdn(). (smohanty)
-
- AMBARI-2223. Using an external MySQL / Oracle database for Oozie does not 
- work. (swagle)
-
- AMBARI-2222. Analyze puppet scripts for potential dependency issues. (swagle)
-
- AMBARI-2221. Simplify JDBC driver setup process. (swagle)
-
- AMBARI-2220. Update deployed log4j conf for Hbase deployments from Ambari. 
- (swagle)
-
- AMBARI-2213. RegionServer gets  exception while accessing staging directory
- in secure cluster. (jaimin) 
-
- AMBARI-2218. Unit test failures. (smohanty)
-
- AMBARI-2217. Increase ambari-agent test coverage. (smohanty)
-
- AMBARI-2211. YARN does not show any configurations. (yusaku) 
-
- AMBARI-2210. Hadoop 2 stack version check should be integer based instead of
- string. (yusaku)
-
- AMBARI-2208. Reassign Master Wizard: refreshing page on step 2, 3 or 4 breaks
- wizard. (yusaku)
-
- AMBARI-2205. Reassign Master Wizard: Installing master with TIMEDOUT should
- not be completed. (yusaku)
- 
- AMBARI-2204. Reassign Master Wizard: Review Config shouldn't show unchanged
- values. (yusaku)
-
- AMBARI-2203. Background operations popup does not automatically refresh the
- task log. (yusaku)
-
- AMBARI-2202. Running the smoke test for a specific service should set the
- appropriate request context showing the service name. (yusaku)
-
- AMBARI-2209. Add monitoring capabilities for Hadoop 2.0. (smohanty)
-
- AMBARI-2201. Hbase master and regionserver should use same keytab. (jaimin)
-
- AMBARI-2162. HBase master stop fails while stopping all services in
- secure cluster. (jaimin)
-
- AMBARI-2197. Jobs page usability issues - remove the "ramps". (srimanth)
-
- AMBARI-2196. Install Wizard, Customize Services changes. (srimanth)
-
- AMBARI-2193. Remove zookeeper dependency for HIVE and HCAT. (srimanth)
-
- AMBARI-2184. Install progress gets stuck at 33% and displays incorrect
- status upon refresh. (smohanty)
-
- AMBARI-2159. After upgrading ambari from 1.2.2.5 to 1.2.3.6 the server
- throws 500 error when starting/stopping any service. (smohanty)
-
- AMBARI-2186. Hadoop2 - Handle dfs include/exclude properties. (srimanth)
-
- AMBARI-2179. Fix JDBC URLs for Oozie connecting to MySQL server. (yusaku)
-
- AMBARI-2178. Fix JDBC URLs for connecting to Oracle from Hive Metastore and
- Oozie Server. (yusaku)
-
- AMBARI-2172. Fix unit tests for Ambari Web. (yusaku)
-
- AMBARI-2171. Host status filter not restored on Hosts page when navigating
- back. (yusaku)
-
- AMBARI-2157. Page title and cluster name shown in the header are not the same
- case. (yusaku)
-
- AMBARI-2183. Postgres setup failed. (smohanty)
-
- AMBARI-2182. Expose "active" attribute for Stack resource. (smohanty)
-
- AMBARI-2181. keys/pass.txt file is readable by everyone. (smohanty)
-
- AMBARI-2177. HBase Smoke Test shows success even if it fails. (smohanty)
-
- AMBARI-2176. Incorrect unit displayed for "dfs.datanode.du.reserved" and
- "fs.checkpoint.size". (jaimin)
-
- AMBARI-2175. Mark 2.0 stack to be active for testing purposes. (srimanth)
-
- AMBARI-2173. TEST BROKEN : FAIL: test_upgradeCommand_executeCommand
- (TestActionQueue.TestActionQueue). (smohanty)
-
- AMBARI-2165. Ambari server upgrade fails when upgrading
- second time. (smohanty)
-
- AMBARI-2166. Stack versions should be marked as active. (srimanth)
-
- AMBARI-2164. START_FAILED and STOP_FAILED no longer exist, the upgrade 
- script should repair hostcomponentstate table to convert these to i
- INSTALLED. (smohanty)
-
- AMBARI-2156. zookeeper service check fails when zk client is not co-hosted
- with zk server. (jaimin)
-
- AMBARI-2161. Datanode Start fails in secure cluster. (jaimin)
-
- AMBARI-2160. zookeeper smoke test fails with unknown host exception for
- secure cluster.(jaimin)
-
- AMBARI-2155. Incorrect page display after closing Reassign Master Wizard.
- (yusaku)
-
- AMBARI-2154. Unwanted auto scrolling on the Host detail page. (yusaku)
-
- AMBARI-2153. Reassign Master Wizard: Background Operations popup should show
- display names for services and components. (yusaku)
-
- AMBARI-2152. Sometimes stale host / host component indicators are shown.
- (yusaku)
-
- AMBARI-2151. Incorrect error message for invalid user name. (yusaku)
-
- AMBARI-2149. Ambari needs to set right path for GC log directory of Hbase
- process. (mahadev)
-
- AMBARI-2116. Add parameters to improve HIVE performance. Missing property. 
- (swagle)
-
- AMBARI-2146. When hive and oozie users have been changed after upgrade hive
- metastore and oozie cannot start properly. (smohanty)
-
- AMBARI-2145. Create script for deploying all new services for UI. (smohanty)
-
- AMBARI-2144. Installation with existing Oracle DB fails. (swagle)
-
- AMBARI-2143. HBASE fails to start on master. (smohanty
-
- AMBARI-2141. When hbase user is changed hbase does not start properly
- after upgrade. (smohanty)
-
- AMBARI-2142. Unecessary use of @Transactional on all DAO methods. (swagle)
-
- AMBARI-2107. Cluster CPU Chart is off the charts. Reopened. (swagle)
-
- AMBARI-2139. Hive Service check fails on secure cluster. (jaimin)
-
- AMBARI-2138. HBase smoke test fails on secure cluster. (swagle)
-
- AMBARI-2136. Home paths are not set correctly in /etc/sqoop/conf/sqoop-env.sh
- (mahadev)
-
- AMBARI-2135. Ambari agent start say to look at .out, not .log. (smohanty)
-
- AMBARI-2134. Set default value of oozie property
- "oozie.service.AuthorizationService.authorization.enabled" to true. (jaimin)
-
- AMBARI-2132. Error during "HDFS" service starting. (swagle)
-
- AMBARI-2116. Add parameters to improve HIVE performance. With decriptions. 
- (swagle)
-
- AMBARI-2127. Service check should not emit log with Exceptions considering
- its a positive scenario. (smohanty)
-
- AMBARI-2124. Hive client is not getting reconfigured when it is co-hosted with 
- hive server/metastore. (swagle)
-
- AMBARI-2114. Refresh on the Service page always activates summary tab. (jaimin)
-
- AMBARI-2120. Add few logs to allow root-cause the issue where agent seems
- to unable to communicate with the server. (smohanty)
-
- AMBARI-2116. Add parameters to improve HIVE performance. (swagle)
-
- AMBARI-2117. Set jobtracker retire interval. (swagle)
-
- AMBARI-2109. Sanitize KEYS and NOTICE.txt on trunk. (yusaku)
- 
- AMBARI-2108. Fix apache rat check issues for ambari (top-level dir).
- (yusaku)
-
- AMBARI-2105. Assign Slaves page allows the user to specify a host with no
- components on it. (jaimin)
-
- AMBARI-2106. Fix apache rat check issues for ambari-server and ambari-agent.
- (swagle)
-
- AMBARI-2107. Cluster CPU Chart is off the charts. (swagle)
-
- AMBARI-2102. Confusing message "ls: cannot access /usr/share/java/*oracle*:
- No such file or directory". (smohanty)
-
- AMBARI-2104. Fix apache rat check issues for ambari-web. (yusaku)
-
- AMBARI-2100. HBase throws AccessDeniedException. (yusaku)
-
- AMBARI-2099. Cluster install failed due to timeout and the user can proceed
- to cluster management; the user was not presented an option to retry install.
- (yusaku)
-
- AMBARI-2101. Hive service check (still) failing with file permissions.
- (swagle)
-
- AMBARI-2095. It's possible to get into a state where install retry is not
- possible if the agent stops heartbeating. (jaimin via yusaku)
-
- AMBARI-2091. Custom JDK path not used when adding new hosts. (yusaku)
-
- AMBARI-2089. Post Ambari upgrade, Hive and Oozie fail to start after
- reconfigure. (Xi Wang via yusaku)
-
- AMBARI-2084. Wrong host mapping in Assign Masters step. (yusaku)
-
- AMBARI-2098. Customizing webcat pid run directory fails service status. 
- (swagle)
-
- AMBARI-2076. DataNode install failed with custom users. (swagle)
-
- AMBARI-2085. UI allows user to set empty value for configs in
- Advanced category. (jaimin)
-
- AMBARI-2087. Tasks are not filtered by parent request id. (smohanty)
-
- AMBARI-2086. Agent on host with clients and DATANODE only seems to schedule 
- STATUS commands for several other services. (swagle)
-
- AMBARI-2088. Cluster installation times out at server side too fast. (swagle)
-
- AMBARI-2083. Upgrade fails on Sles. (smohanty)
-
- AMBARI-2082. Oozie service check fails. (jaimin)
-
- AMBARI-2081. changeUid.sh failing during installation. (swagle)
-
- AMBARI-2079. Can't change service configuration if heartbeat lost from
- service component host. (yusaku)
-
- AMBARI-2075. Admin role can't be assigned to LDAP user. (yusaku)
-
- AMBARI-2080. Cluster name and Background operations indicator should
- disappear on logout. (jaimin)
-
- AMBARI-2078. Hive Metastore host not changing on Assign Masters page. (jaimin)
-
- AMBARI-2077. Update stack mock data to make testMode functional on step4 of
- installer wizard. (jaimin)
-
- AMBARI-2076. DataNode install failed with custom users. (smohanty)
-
- AMBARI-2074. Deployment of HDP 1.2.1 fails on Sles. (smohanty)
-
- AMBARI-2073. After Ambari upgrade to 1.2.3, MapReduce service check fails 
- because uid of ambari_qa changed. (swagle)
-
- AMBARI-2067. hive-site.xml cannot be readonly for clients. (swagle)
-
- AMBARI-2068. "Preparing to install <component>" message needs spacing.
- (yusaku)
-
- AMBARI-1979. Last HeartBeat time and heartbeat status for agent take around 2-3 
- minutes to update on a server restart. (swagle)
-
- AMBARI-1983. Add new parameters to improve HBase MTTR. HDPLocal fixes. 
- (swagle)
-
- AMBARI-2066. HDFS shortcircuit skip checksum should be removed. (smohanty)
-
- AMBARI-2056. Show proper error message while user tries to save configurations 
- of partially stopped service. (srimanth)
-
- AMBARI-2064. Legend for zoomed-in graphs do not render properly in IE9.
- (yusaku)
-
- AMBARI-2063. Admin features not available for user with admin rights under
- certain conditions. (yusaku)
-
- AMBARI-2060. Initiate a recommission, on success, the operations dialog says
- decommission, not recommission. (yusaku)
-
- AMBARI-2058. Host Detail page: if the host component is in INSTALL_FAILED
- state, we should let the user reinstall it. (yusaku)
-
- AMBARI-2055. Oozie reconfig forces the user to enter bogus values for two
- parameters in order to save any changes. (yusaku)
-
- AMBARI-2054. If "Install from Local Repository" selected in install wizard,
- Add Host wizard not working. (yusaku)
-
- AMBARI-2053. Align "add hosts" button vertically with host health filter.
- (yusaku)
-
- AMBARI-2052. Fix delete user popup. (yusaku)
-
- AMBARI-2065. Hadoop group customization does not take affect. (smohanty)
-
- AMBARI-2062. Service versions shown during install dont match installed
- versions. (smohanty)
-
- AMBARI-2038. Services links on Dashboard connected to incorrect pages.
- (yusaku)
-
- AMBARI-2059. Add dependency for Nagios server on Hive Client install. (swagle)
-
- AMBARI-2044. hive-site.xml permission denied exception. (swagle)
-
- AMBARI-2057. Gmond left in init after install. (smohanty)
-
- AMBARI-2051. Remove hard-coded ports from agent scripts - Nagios. (swagle)
-
- AMBARI-2045. Add Unit test to verify, client re-install for install failed 
- client. (swagle)
-
- AMBARI-2044. hive-site.xml permission denied exception. (swagle)
-
- AMBARI-2041. If a host that has a service client installed and the host is down, 
- service start will fail. (swagle)
-
- AMBARI-2039. Service check should be scheduled on a client that is on
- a host in HEALTHY state - use correct state enum. (smohanty)
-
- AMBARI-2035. "Add local user" button is enabled but nothing happens upon
- clicking it under certain conditions. (yusaku)
-
- AMBARI-2034. Disable "Add Component" button in the Host Details page if the
- host is in UNKNOWN state or !isHeartbeating. (yusaku)
-
- AMBARI-2033. Decommission DataNode does not have any request context.
- (yusaku)
-
- AMBARI-2029. Error when loading /main/services directly. (yusaku)
- 
- AMBARI-2039. Service check should be scheduled on a client that is on
- a host in HEALTHY state. (smohanty)
-
- AMBARI-2037. Nagios web not installing as expected on Sles11. (swagle)
-
- AMBARI-1924. Allow for users to customize Ganglia gmetad + gmond user
- accounts. (smohanty)
-
- AMBARI-2024. Ambari Server becomes unresponsive after crashing on http reads 
- on jersey. (swagle)
-
- AMBARI-2020. Incorrect behavior of "Services" page. (yusaku)
-
- AMBARI-2018. Hosts page: no filter selection is shown after clicking on
- "Alerts" filter, navigating away, and coming back to Hosts page. (yusaku)
-
- AMBARI-2016. Hide Maintenance pulldown if no operation can be performed.
- (yusaku)
-
- AMBARI-2015. Host component start/stop causes "Uncaught TypeError: Cannot call
- method 'call' of undefined". (yusaku)
-
- AMBARI-2011. Add Hosts gets stuck at 33% (some hosts in the cluster were
- down). (yusaku)
-
- AMBARI-2014. Install Wizard/Add Host Wizard Review page: local repo option
- is always displayed as "No", even when it is enabled. (yusaku)
-
- AMBARI-2019. Cannot decommission data node (ensure recommission also works).
- (swagle)
- 
- AMBARI-2021. Hadoop installation on cluster with SUSE-11 failed. (smohanty)
-
- AMBARI-2010. Tasks do not timeout for failed hosts. (swagle)
-
- AMBARI-2012. Check Ambari-agent process - nagios alert is only being
- configured on the nagios-server host. (smohanty)
-
- AMBARI-2001. Filtering on Jobs table does not work under certain situations.
- (yusaku)
-
- AMBARI-2000. Undo links still remain after the config changes are saved.
- (yusaku)
-
- AMBARI-1999. Clicking on Cancel on the Service Config page should not reload
- the entire app. (yusaku)
-
- AMBARI-1998. Action buttons on host details page not formatted properly on
- Firefox. (yusaku)
-
- AMBARI-1997. Filtered hosts get out of sync with the filter selection. (yusaku)
-
- AMBARI-2009. task-log4j.properties file ownership should not be
- root. (smohanty)
-
- AMBARI-2008. Using mixed OS overwrites ambari.repo during install. (smohanty)
-
- AMBARI-1952. hadoop dependency version for ambari-log4j is hardcoded, making
- it regular expression based to pick latest from the repository. (smohanty)
-
- AMBARI-2007. Decom DataNode throws JS error. (smohanty)
-
- AMBARI-1994. Adding component to Host should should wire-up + adjust
- associated Nagios alerts. (smohanty)
-
- AMBARI-1753. Puppet paramter configuration not working as expected. (swagle)
-
- AMBARI-1978. Deploying HDP-1.3.0 results in several alerts - is it related to 
- hard-coded port. Incremental update. (swagle)
-
- AMBARI-1990. After successful registration, going back to the Confirm Hosts
- or re-installing agents from Install Options page causes host registration
- to fail. (smohanty)
-
- AMBARI-1991. Remove unused python files from ambari-agent. (smohanty)
-
- AMBARI-1984. WebHCat log and pid dirs configs should be under WebHCat >
- Advanced. (yusaku)
-
- AMBARI-1989. Add component shows the same component again even if the
- component is already added/installed/started. (yusaku)
-
- AMBARI-1988. Hostname pattern expression is broken. (yusaku)
-
- AMBARI-1986. HDFS General section has disappeared from Customize Services 
- step of the Install Wizard. (yusaku)
-
- AMBARI-1985. Incorrect behavior of "Undo" button for password fields. (yusaku)
-
- AMBARI-1702. Ambari/GSInstallers need to set the value of 
- mapred.jobtracker.completeuserjobs.maximum. New recommended value. (swagle)
-
- AMBARI-1983. Add new parameters to improve HBase MTTR. (swagle)
-
- AMBARI-1979. Last HeartBeat time and heartbeat status for agent take around 2-3 
- minutes to update on a server restart. (swagle)
-
- AMBARI-1978. Deploying HDP-1.3.0 results in several alerts - is it related to 
- hard-coded port. (swagle)
-
- AMBARI-1974. BootStrapTest is failing on the master build. (smohanty)
-
- AMBARI-1968. Hadoop Classpath is being overwridden which causes hive
- server/metastore to fail. (smohanty)
-
- AMBARI-1973. log4j Appender for RCA should be able to write the same database
- being used for Ambari Server (oracle/MySql). (smohanty)
-
- AMBARI-1972. Stacks2 api implemenation using the standard framework is not
- complete - does not show configuration tags. (smohanty)
-
- AMBARI-1954. Dashboard does not come up if the upgrade stack does not contain
- a service with the same name. (yusaku)
-
- AMBARI-1953. On Add Hosts, the request context for the start phase shows up
- as "Request Name Not Specified". (yusaku)
-
- AMBARI-1966. Client install tasks are shown twice in progress popup during
- start phase of install wizard (update API call to include
- params/reconfigure_client). (yusaku)
-
- AMBARI-1965. core-site properties are incorrectly populated in Advanced/
- General category of MapReduce service. (yusaku)
-
- AMBARI-1963. Deploying progress bar shows 0 tasks after installation failure
- and going back to a previous step to retry. (yusaku)
-
- AMBARI-1962. Host Check popup keeps the "rerun check" button disabled even
- after it is done and its hard to know if its actually run or not. (yusaku)
-
- AMBARI-1961. Select Services: clicking on "all" selects HUE even when HUE
- support is toggled off. (yusaku)
-
- AMBARI-1960. "Back" button can be pressed while host registration is taking
- process, even though the button seems disabled. (yusaku)
-
- AMBARI-1959. Cannot login to Ambari after login failure. (yusaku)
-
- AMBARI-1957. Hosts table: whether the alert filter is in effect or not is
- not clear. (yusaku)
-
- AMBARI-1956. Wrong install status shown in Add Service Wizard. (yusaku)
-
- AMBARI-1951. Ambari agent setup during bootstrap should install the same
- version of agent as the server. (smohanty)
-
- AMBARI-1950. Hadoop install was failed on SUSE-11.1sp1 cluster with all 
- services except Hue. (smohanty)
-
- AMBARI-1949. Reconfiguration of Services has issues and the configurations 
- save button does not take affect. (srimanth)
-
- AMBARI-1948. System logs are not present on tasktracker. (swagle)
-
- AMBARI-1947. Oozie Smoke test fails with errors on the start services/install 
- page. (swagle)
-
- AMBARI-1946. Heatmap memory should not include cached memory as part of
- "used". (Jeff Sposetti via yusaku)
-
- AMBARI-1944. All Service Smoke tests fail when run with service start. (swagle)
-
- AMBARI-1939. Make service restart feedback based on supports functionality. 
- (srimanth)
-
- AMBARI-1943. Properties that do not map to any global property are not being
- sent to server. (jaimin)
-
- AMBARI-1937. Ambari-web installer wizard doesn't work in test mode. (jaimin)
-
- AMBARI-1927. In background operations popup, requests with same context
- are showing hosts/tasks info from last request. (yusaku via jaimin)
-
- AMBARI-1907. Service check commands are not getting created on a
- cluster install -> start. (yusaku via jaimin)
-
- AMBARI-1942. Nagios server failed to start. (swagle)
-
- AMBARI-1938. Update mock data for stack HDP-1.3.0. (jaimin)
-
- AMBARI-1934. Security vulnerability with Ganglia and Nagios. (smohanty)
-
- AMBARI-1933. Test failure : testCascadeDeleteStages. (smohanty)
-
- AMBARI-1931. params/run_smoke_test=true is not taking effect. (smohanty)
-
- AMBARI-1919. JobTracker History Server failed to come up on 1.3.0 stack
- and the request for service stall is stalled. (smohanty)
-
- AMBARI-1900. Update the DDL update script to modify the table to includei
- ph_cpu_count. (smohanty)
-
- AMBARI-1926. One HBase master should have active HA status at all time. 
- (smohanty)
-
- AMBARI-1925. Remove "hadoop_deploy" user. (smohanty)
-
- AMBARI-1915. Client install tasks are shown twice in install progress 
- popup. (swagle)
-
- AMBARI-1916. Filter for showing only properties which need restart is 
- broken. (srimanth)
-
- AMBARI-1918. Set correct Oozie property for security instead of deprecated
- property. (jaimin)
-
- AMBARI-1917. Ambari Core-Site.xml Missing Property for LZO (enabled) -
- io.compression.codecs (jaimin).
-
- AMBARI-1889. Added documentation for configuration (ncole)
-
- AMBARI-1912: HBase master doesn't come up after disabling security. (jaimin)
-
- AMBARI-1902: RegionServer does not start in secure cluster. (jaimin)
-
- AMBARI-1903. Host Exception Popup layout and cosmetic issues. (srimanth)
-
- AMBARI-1901. Add additional tests for verifying request behavior based on 
- host role command results. (smohanty)
-
- AMBARI-1899. ambari-reset does not respect -s. (swagle)
-
- AMBARI-1898. Update stack definitions for 1.3.0. (smohanty)
-
- AMBARI-1886. Derived properties not being overridden for hosts. (srimanth)
-
- AMBARI-1896. Disable editing Capacity Scheduler on host configs. (srimanth)
-
- AMBARI-1894. Refactor configs of Capacity Scheduler category. (srimanth)
-
- AMBARI-1893. Parsing new alerts format fails. (srimanth)
-
- AMBARI-1891. Impossibility to scroll metric window after browser width 
- changing. (srimanth)
-
- AMBARI-1880. stacks2 API uses "type" to refer to config tags and no longer
- exposes "filename" as a property. (srimanth via yusaku)
-
- AMBARI-1873. HUE pid and log dir labels are flip flopped. (yusaku)
-
- AMBARI-1878. Host overrides functionality broken in wizard Step7 controller.
- (yusaku)
-
- AMBARI-1875. Restart Service tooltip overlaps another tooltip. (yusaku)
-
- AMBARI-1874. Add Service Wizard: remove the ability to install master
- components for already installed services. (yusaku)
-
- AMBARI-1872. Ambari FE is not setting proper value for 
- fs.checkpoint.edits.dir (jaimin)
-
- AMBARI-1869. Permission on agent site.pp files needs to be 660. (swagle)
-
- AMBARI-1867. Processing API requests takes too long. (swagle)
-
- AMBARI-1856. Queries for metrics to populate the dashboard graphs don't work
- with updated Ganglia. (tbeerbower)
-
- AMBARI-1862. Nagios credentials are freely available at ambari-agent.log.
- (smohanty)
-
- AMBARI-1726. It seems upgrades available at the FE is hard-coded to 1.3.0. 
- (yusaku via srimanth)
-
- AMBARI-1854. Wizards available for a non-administrator user. (yusaku via srimanth)
-
- AMBARI-1852. Upon clicking Services > Service > Config, a call to 
- "configurations resource is made and the server throws 400. (yusaku via srimanth)
-
- AMBARI-1851. Ambari Web behaves strangely when there is no Active HBase 
- Master. (yusaku via srimanth)
-
- AMBARI-1849. Cosmetic problems on HBase Dashboard. (yusaku via srimanth)
-
- AMBARI-1848. Install Wizard, Step 7: Oozie Database Derby option should say 
- "New Derby Database", not "Current Derby Database". (Xi Wang via srimanth)
-
- AMBARI-1860. Master broken - Cannot deploy services. (smohanty)
-
- AMBARI-1859. Cannot load Nagios Alerts due to 400 Bad Request. (smohanty)
-
- AMBARI-1842. Collapsable service restart message section should have pointer 
- cursor. (srimanth)
-
- AMBARI-1841. Properties that should be exposed in Advanced category
- are populated in Custom categories. (jaimin)
-
- AMBARI-1837. Few core-site properties vanished after seemingly benign 
- reconfiguration. (jaimin)
-
- AMBARI-1838. Cluster Management > Services > MapReduce > Config throws JS error
- and the page comes up blank. (jaimin)
-
- AMBARI-1836. Remove hard-coded ports from agent scripts. (swagle)
-
- AMBARI-1834. Reduce the number of states that a host component can be in.
- (smohanty)
-
- AMBARI-1789. Stopping and then Starting all services doesn't start 
- NameNode. (smohanty)
-
- AMBARI-1822. Hue service link points to wrong URL and no smoke test drop
- down is shown. (yusaku)
-
- AMBARI-1821. Upgrading component is not very clear and Upgrade action
- is not available. (yusaku)
-
- AMBARI-1820. Installer Step 7 - DataNode hosts, TaskTracker hosts, and
- RegionServer hosts not displayed correctly. (yusaku)
-
- AMBARI-1819. Ambari Installer: page refreshes upon hitting enter in text
- fields (Step 1 and Step 7). (yusaku)
-
- AMBARI-1813. The back button seems disabled during host registration (step 3),
- but you can actually click it to go back. (yusaku)
-
- AMBARI-1812. Unable to re-configure core-site. (yusaku)
-
- AMBARI-1811. Start/Stop service doesn't work. (yusaku)
-
- AMBARI-1810. Security Wizard - Progress popup is not filtering tasks
- correctly. (yusaku)
-
- AMBARI-1806. Maintenance checks issued from frontend does not have request
- context set appropriately. (yusaku)
-
- AMBARI-1804. Reassign master should show only the hosts that do not have
- another instance of the master for HBase. (yusaku)
-
- AMBARI-1803. Reassign HBase master menu displays multiple entries with no
- distinction when there are multiple HBase masters. (yusaku)
-
- AMBARI-1802. Install wizard and subsequent reconfig screens lose 'confirm'
- password content and show up as red even if the use is not editing these
- fields. (yusaku)
-
- AMBARI-1801. After adding hosts successfully, you need to refresh the hosts
- page manually to see the new hosts. (yusaku)
-
- AMBARI-1799. On service reconfig, Save button can be clicked even when there
- are validation errors. (yusaku)
-
- AMBARI-1796. Specific custom configs do not display after reload.
- (srimanth via yusaku)
-
- AMBARI-1768. Cluster install wizard does not succeed at service start.
- (yusaku)
-
- AMBARI-1755. Provide context for background operations. (srimanth via yusaku)
-
- AMBARI-1744. isAdmin doesn't switch after login/out. (srimanth via yusaku)
-
- AMBARI-1709. When all hosts are assigned a master component, the last host
- should have all slave components and clients (Step 6). (srimanth via yusaku)
-
- AMBARI-1695. Customize Services page - validation error count is not reflect
- in the service tab for host exceptions. (yusaku)
-
- AMBARI-1675. ASF license header missing from
- app/templates/main/admin/security/add/step2.hbs. (yusaku)
-
- AMBARI-1670. Changing service user name from web UI should also change
- configuration properties that depends on those user name. (jaimin via yusaku)
-
- AMBARI-1826. Use service stop and start for Nagios/Ganglia/MySQL rather than
- puppet artifacts for starting stopping these services. (smohanty)
-
- AMBARI-1818. HBase master shuts down immediately after start in a secure 
- cluster. (swagle)
-
- AMBARI-1816. Security wizard: Add missing secure configs to Hbase service and
- make "zookeeper" as default primary name for zookeeper principal. (jaimin)
-
- AMBARI-1791. Can not specify request context for smoke test request. (swagle)
-
- AMBARI-1788. JMX getSpec error filling up server logs. (swagle)
-
- AMBARI-1787. Nagios script causes Datanode error. (swagle)
- 
- AMBARI-1674. Jobtracker metric for maps_completed shows wrong value
- (tbeerbower)
-
- AMBARI-1786. Ambari server start fail after reset. (smohanty)
-
- AMBARI-1784. MapReduce service damaging after hadoop installation with 
- custom MapReduce user which contains symbol '-'. (smohanty)
-
- AMBARI-1774. Ambari does not push the config updates to the client/gateway 
- node. (swagle)
-
- AMBARI-1780. POSTing new cluster returns 500 exception. (smohanty)
-
- AMBARI-1781. Ambari Server should work with MySQL and Oracle where the 
- Ambari Server data might be stored. (smohanty)
-
- AMBARI-1775. Security wizard - Javascript error is thrown when zooKeeper
- is included as a secure service. (jaimin)
-
- AMBARI-1771. On clicking master component host on Oozie and Hive
- service page javascript error is encountered.(jaimin)
-
- AMBARI-1767. Add ability to customize "ambari_qa" user. (smohanty)
-
- AMBARI-1770. Hue installation fails due to manifest errors. (swagle)
-
- AMBARI-1764. Unable to get all tasks from more than one request_id by one
- request (tbeerbower)
-
- AMBARI-1766. Hide Java Home option on step-7 of Installer wizard. (jaimin)
-
- AMBARI-1765. Enable the Ganglia rrd files location to be configurable
- when Ganglia is selected as service. (jaimin)
-
- AMBARI-1762. SUSE_Unable to start hive. (swagle)
-
- AMBARI-1761. Update the DDL update script to modify the table to
- include ph_cpu_count. (smohanty)
-
- AMBARI-1759. Error in creating host component. (smohanty)
-
- AMBARI-1757. Add support for Stack 1.2.2 to Ambari. (smohanty)
-
- AMBARI-1749. Set default heap size for zookeeper. (swagle)
-
- AMBARI-1748. JDK option on the UI when used is not passed onto the global 
- parameters. (srimanth)
-
- AMBARI-1747. Added executable permission of generate monitor/server scripts.
- (smohanty)
-
- AMBARI-1747. File ownership needs more consistency for those installations 
- where root access is hard to get. (smohanty)
-
- AMBARI-1561. API should return nagios_alerts as a JSON, not a stringified 
- JSON. (smohanty)
-
- AMBARI-1507. Should not install HDPHBaseMaster, HDPNameNode and HDPJobTracker
- ganglia configs on every node. (smohanty)
-
- AMBARI-1746. Backend support for LDAP Group to Ambari Role Mapping. 
- (smohanty)
-
- AMBARI-1506. Installs HBase ganglia configs when HBase not installed.
- (smohanty)
-
- AMBARI-1739. HBase and Zk failed to start on secure install. (swagle)
-
- AMBARI-1732. Oozie service check fails in secure cluster. (jaimin)
-
- AMBARI 1733. Add service/component specific upgrade puppet files. (smohanty)
-
- AMBARI-1731. WebHcat smoke test fails for the secure cluster. (jaimin)
-
- AMBARI-1730. Hive Service check fails in non secure cluster. (jaimin)
-
- AMBARI-1724. Agent has it hard-coded that HDP repo file can only be 
- downloaded once. (smohanty)
-
- AMBARI-1715. Ambari Agent Unit Test Failure: TestFileUtil.py. (smohanty)
-
- AMBARI-1533. Add Nagios check for ambari-agent process for each host in 
- the cluster. (smohanty)
-
- AMBARI-1713. Need to delete private ssh key from /var/run/ambari-server
- /bootstrap/* on Ambari Server after bootstrap is complete. (swagle)
-
- AMBARI-1711. Trunk is broken due  to invalid argument to a puppet custom 
- function hdp_default. (swagle)
-
- AMBARI-1706. Security wizard: "Done" and "back" buttons on Apply step 
- should be disabled while step is in progress. (jaimin)
-
- AMBARI-1705. Remove redundant API calls to update service configuration
- while disabling security. (jaimin)
-
- AMBARI-1661. For custom advanced properties, a new config with an empty key
- can be added. (yusaku)
-
- AMBARI-1659. Arrows often do not show up on config category expander. (yusaku)
-
- AMBARI-1645. Undo should not be allowed on component hosts. (yusaku)
-
- AMBARI-1644. Service summary page flickers. (yusaku)
-
- AMBARI-1689. 500 Exception creating service component during install. (Sumit 
- Mohanty via swagle)
-
- AMBARI-1504. Hosts show physical CPUs, instead of cores. (Sumit Mohanty 
- via swagle)
-
- AMBARI-1685. Remove running of smoke tests by default when services or 
- master components are started. (Sumit Mohanty via swagle)
-
- AMBARI-1688. API support to return 10 most recent requests. (swagle)
-
- AMBARI-1439. rrd file location should be read from global config. 
- New patch for reopened bug. (swagle)
-
- AMBARI-1667. Starting all services fails on secure cluster (excluding 
- HBase and ZooKeeper). (swagle)
-
- AMBARI-1666. Oozie properties for principal and keytab not read from 
- oozie-site. (swagle)
-
- AMBARI-1660. Server seems to ignore failures if the prior stage has failed 
- before the next iteration of the scheduler. (Sumit Mohanty via swagle)
-
- AMBARI-1657. User directories on HDFS do not get created with custom names 
- provided from Ambari UI. (swagle)
-
- AMBARI-2192. Fix for agent hanging during install. (ncole)
-
- AMBARI-2125. Fix for UNKNOWN state failing retry on install. (ncole)
-
- AMBARI-2072. Fix to remove actual_configs from cluster response. (ncole)
-
- AMBARI-2036. Fix to send global configs with status_commands to agents. (ncole)
-
- AMBARI-2025. Fix to restrict how UNKNOWN is assigned to a host-component. (ncole)
-
- AMBARI-2013. Fix to delete cluster with components in unknown state. (ncole)
-
- AMBARI-1977. Honor service configs when there are no matching cluster configs (ncole)
-
- AMBARI-1976. When host expires, update each component for host with unknown state. (ncole)
-
- AMBARI-1980. Fix for nagios_alerts element when there is an error. (ncole)
-
- AMBARI-1865. Fix for upgrade script to copy configurations. (ncole)
-
- AMBARI-1703. Fix for smoke tests getting configurations. (ncole)
-
- AMBARI-1678. Fix when there are no service overrides. (ncole)
-
- AMBARI-1655. DELETE is not successful against ClusterStateEntity (ncole)
-
- AMBARI-1439. rrd file location should be read from global config. (swagle)
-
- AMBARI-1648. Hue configuration - DB properties cannot be empty. (swagle)
-
- AMBARI-1641. Some map and reduce task metrics are missing for the
- tasktrackers in the API. (tbeerbower)
-
- AMBARI-1640. Erroneos property is not highlighted while landing on step7
- of Installer wizard. (jaimin)
-
- AMBARI-1637. JCE test for policy files fails during secure install. (swagle)
-
- AMBARI-1621. Config/Reconfig UI should not allow certain configs to have
- host-level overrides. (yusaku)
-
- AMBARI-1597. Templeton smoke test fails for secure cluster. (swagle)
-
- AMBARI-1600. Make component naming consistent. (yusaku)
-
- AMBARI-1625. Oozie start fails on secure cluster. (swagle)
-
- AMBARI-1627. Fix to remove host configuration overrides. (ncole)
-
- AMBARI-1592. Fix configuration propagation.
-
- AMBARI-1619. Fix for category path separators.
-
- AMBARI-1616. Error during upgrading Ambari Server from 1.2.0/1.2.1 to 
- 1.2.2. (Sumit Mohanty via swagle)
-
- AMBARI-1603. JCE install on ambari-server fails if /tmp/HDP-artifacts does 
- not exists. (swagle)
-
- AMBARI-1612. Parameterizing nagios and ganglia monitoring rpm version.
- (Ashish Singh via yusaku)
-
- AMBARI-1586. Upgrade of Ambari DB on upgrade to 1.2.2 should restore/keep 
- the configuration data for MAPREDUCE. (Sumit Mohanty via swagle)
-
- AMBARI-1594. Ambari UI shows failed services while processes are running 
- on the server. (swagle) 
-
- AMBARI-1582. Cannot start hadoop services after hdfs re-configuration 
- and amabri server restart. (swagle)
-
- AMBARI-1570. Dashboard - missing translations. (Xi Wang via yusaku)
-
- AMBARI-1569. Add AMBARI-1536 and 1537 back. (Xi Wang via yusaku)
-
- AMBARI-1579. Admin page side nav does not use the correct style and does
- not highlight selection. (yusaku)
-
- AMBARI-1552. Missing translations on Dashboard. (Xi Wang via yusaku)
-
- AMBARI-1549. Cluster name displayed incorrectly in the top nav. (Xi Wang
- via yusaku) 
-
- AMBARI-1559. Jobs failed count always returns 0 in the jobtracker API metrics.
- (tbeerbower)
-
- AMBARI-1577. Apply stage of security wizard throws javascript error 
- for loadStep function. (jaimin)
-
- AMBARI-1575. Service should be shown red when any of its master component 
- is in START_FAILED status. (jaimin)
-
- AMBARI-1565. Ambari server throws EntityExistsException on transitioning 
- from INIT to INSTALLED state. (Sumit Mohanty via swagle)
-
- AMBARI-1564. TestActionManager is failing on master branch. (swagle)
-
- AMBARI-1561. API should return nagios_alerts as a JSON, not a 
- stringified JSON. (swagle)
-
- AMBARI-1492. Add init.d scripts for Ambari server + agent. (swagle)
-
- AMBARI-1548. Implement Stacks API using the consistent API framework in 
- Ambari Server with all the get/predicates working. (swagle)
-
- AMBARI-1544. AmbariManagementControllerTest has extra import that cannot 
- be resolved. (swagle)
-
- AMBARI-1539. Stage creation takes on an average 1.5 minutes on large 
- cluster. (swagle) 
-
- AMBARI-1485. Server throws exception when trying to stop a service which is 
- in stopping state. (swagle) 
-
- AMBARI-1526. State fields are not returned by default for services.
- (tbeerbower)
-
- AMBARI-1527. Allow loading of custom configurations in
- step7 : "Customize Services" (regression). (jaimin)
-
- AMBARI-1525. ambari.properties file has an invalid character causing
- ambari-server setup to fail. (jaimin)
-
- AMBARI-1524. Service summary for Hive does not display clients properly.
- (yusaku)
-
- AMBARI-1520. Alerts take around 20-30 seconds to show up everytime you
- refresh the dashboad. (srimanth via yusaku)
-
- AMBARI-1523. Ambari API: Resources doesn't always honor partial response
- fields restrictions. (tbeerbower)
-
- AMBARI-1519. Ambari Web goes back and forth between frozen and usable state
- peridocially on a large cluster. (yusaku)
-
- AMBARI-1499. Add hosts is broken. (yusaku)
-
- AMBARI-1501. Nagios alerts do not update automatically. (yusaku)
-
- AMBARI-1503. Ajax call sometimes adds multiple question marks in the query
- causing bad requests. (yusaku)
-
- AMBARI-1463. State of HBase region server not updated when instance is shut
- down on a cluster not installed via Ambari. (tbeerbower)
- 
- AMBARI-1494. Browser memory consumption issues. (jaimin)
-
- AMBARI-1480. Comparison predicate should account for null values. (tbeerbower)
-
- AMBARI-1467. UI should block on cluster metric api call before making
- subsequent one. (yusaku)
-
- AMBARI-1462. PB (petabytes) is shown as "undefined". (yusaku)
-
- AMBARI-1455. Setting App.testMode=true, alwaysGoToInstaller=true does not
- render the Dashboard properly upon login. (yusaku)
-
- AMBARI-1452. Graphs look broken when network connectivity is lost between
- Ambari Web and Ambari Server. (yusaku)
-
- AMBARI-1441. Validation for username used in service configs is broken.
- (yusaku)
-
- AMBARI-1456. Cannot proceed after bootstrapping in some cases due to a
- run-time error while running host checks. (yusaku)
-
- AMBARI-1449. Failure popup shown for reconfiguring HDFS when MapReduce 
- is not selected. (jaimin)
-
- AMBARI-1445. Redirect to main app page when testMode flag is set True and
- alwaysGoToInstaller flag is set False. (jaimin)
-
- AMBARI-1439. rrd file location should be configurable through UI. (jaimin)
-
- AMBARI-1479. Query Lexer sometimes fails to properly parse query strings with
-              ignored properties such as 'fields' present. (jspeidel)
- 
- AMBARI-1446. URL used by API to invoke Ganglia rrd script may exceed max length 
-              for query string for large clusters. (jspeidel)
-
- AMBARI-1431. Hosts table no longer allows sorting. (yusaku)
-
- AMBARI-1376. Wrong calculation of duration filter on apps page. (jaimin via
- yusaku)
-
- AMBARI-1165. Change the dashboard graph for HBase since its using cumulative
- metrics. (yusaku)
-
- AMBARI-1372. three sorting states on jobs table. (jaimin)
- 
- AMBARI-1350. UI screen shifts left-right depending on scrollbar. (jaimin)
-
- AMBARI-1367. Job# for Mapreduce jobs is seen as x. (jaimin)
-
- AMBARI-1363. Graphs jump around upon loading. (jaimin)
-
- AMBARI-1362. Alerts for the hosts with ZooKeeper Server grows on every poll. (jaimin)
-
- AMBARI-1360. Mouse cursor hover behavior is strange on Job Browser. (jaimin) 
-
- AMBARI-1359. App Browser rows colours should alternate from dark grey to light 
- grey and back. (jaimin)
-
- AMBARI-1356. Error in filtering Configuration properties maintained at UI for 
- WebHcat service. (jaimin)
-
- AMBARI-1352. Host-level alert badges should only show the total number
- of CRIT and WARN alerts for the host excluding OK. (jaimin)
-
- AMBARI-1355. Inconsistent casing and component name for alert title. (jaimin)
-
- AMBARI-1354. "No alerts" badge on the Host Detail page should be green, not red. (jaimin)
-
- AMBARI-1353. "Missing translation" shown in Job Browser. (jaimin)
-
- AMBARI-1351. Provide consistent ordering of hosts in heatmap. (jaimin)
-
- AMBARI_1344. mapred.tasktracker.reduce.tasks.maximum in mapred-site.xml is not
- taking effect. (yusaku)
-
- AMBARI-1345. Alerts are not showing up at all in Service pages. (yusaku)
-
- AMBARI-1346. The number of task trackers does not reflect the actual number
- in MapReduce service summary after new TaskTrackers have been added until
- page refresh. (yusaku)
-
- AMBARI-1331. Step 8 hangs on deploy task 2 of 59, server has exception (tbeerbower)
-
- AMBARI-1164. Disk info ganglia metrics is broken for some OS. (Dmytro Shkvyra via jspeidel)
-
- AMBARI-1325. Left border is missing from the main nav. (srimanth)
- 
- AMBARI-1324. Job Browser default sort order should be Run Date DESC. (srimanth)
- 
- AMBARI-1323. Job Browser's column sizing needs to be improved on Firefox. (srimanth)
-
- AMBARI-1321. Switching out of Jobs page does not launch popup anymore.
- (srimanth via yusaku) 
-
- AMBARI-1313. Alert time jumps between 'less than a minute ago' and 'about a
- minute ago'. (srimanth via yusaku) 
-
- AMBARI-1304. When switching jobs in timeline + tasks charts, blank charts show.
- (Arun Kandregula via yusaku) 
-
- AMBARI-1317. Deploy progress returns to deploy screen (momentarily).
- (Arun Kandregula via yusaku) 
-
- AMBARI-1316. Vertical scrollbar shows regardless of how tall the browser height
- is (content height is always slightly taller than viewport). (Arun Kandregula
- via yusaku)
-
- AMBARI-1315. Inconsistent error/warning status in Deploy step; install
- stalls. (Arun Kandregula via yusaku)
-
- AMBARI-1281. Heatmap does not show up if the cluster was installed by going
- back to a previous step from the Deploy step after an install failure.
- (yusaku)
-
- AMBARI-1300. Service status / host component status can get stuck in the
- green blinking state if stop fails - no further operation can be performed.
- (srimanth via yusaku) 
-
- AMBARI-1297. Edit User: if "old password" is not specified and "new
- password" is specified, password update silently fails. (Jaimin Jetly via
- yusaku)
-
- AMBARI-1282. Admin user can lose its own admin privilege. (Jaimin Jetly
- via yusaku)
-
- AMBARI-1292. Add hosts should skip host checks on existing list of cluster
- nodes. (srimanth via yusaku)
-
- AMBARI-1290. Left border is missing from the summary section on Jobs page.
- (srimanth via yusaku)
-
- AMBARI-1278. Cannot proceed from Step 3 to Step 4 in App.testMode (Next 
- button is disabled). (srimanth)
-
- AMBARI-1276. Job Graphs need to show x-axis ticks for elapsed time since 
- submission. (srimanth)
-
- AMBARI-1275. Incorrect displaying "Background operations" window after 
- changing state of component. (srimanth)
-
- AMBARI-1273. Edit User: No error message is shown when the user does not 
- enter the correct "old password". (srimanth)
-
- AMBARI-1172. Alert status change does not change time for the alerts.
- (srimanth via yusaku) 
-
- AMBARI-1264. Service graphs refresh with spinners. (yusaku)
-
- AMBARI-1257. Separator missing in between Oozie and ZooKeeper. (yusaku)
-
- AMBARI-1251. Fix routing issues on Add Host Wizard. (yusaku)
-
- AMBARI-1230. There is a big gap in the lower part of the Jobs table header.
- (yusaku)
-
- AMBARI-1212. After successful install with Ambari, the user is taken to the
- welcome page of the Install Wizard upon browser relaunch if the HTTP session
- is expired. (yusaku)
-
- AMBARI-1227. Host-level task popup is not showing the display name for
- components. (yusaku)
-
- AMBARI-1226. On Dashboard, links to host components are missing. (yusaku)
-
- AMBARI-1219. After adding hosts, the number of live TaskTrackers is not
- updated. (yusaku)
-
- AMBARI-1176. In some cases, once Add Hosts wizard has run once, it requires
- a log out before the Add Hosts wizard can be run again. (yusaku)
-
- AMBARI-1203. mapred-site.xml default system directory is not set
- to /mapred/system. (yusaku)
-
- AMBARI-1200. On some clusters, Nagios alerts show up about 30 seconds after
- page load, while on others the alerts show up immediately. (srimanth via
- yusaku)
-
- AMBARI-1190. Detailed log view dialogs are not center-aligned. (yusaku)
-
- AMBARI-1187. Dashboard > MapReduce mini chart sometimes shows partial graph and hides recent data. (yusaku)
-
- AMBARI-1184. After adding hosts, the host count shown in the Dashboard is
- incorrect. (yusaku)
-
- AMBARI-1178. Fix use of use ip address for JMX metrics request. (tbeerbower
- via mahadev)
-
- AMBARI-1191. Datatable API needs work. (Billie Rinaldi via mahadev)
-
- AMBARI-1211. Ability to configure the same username for all the services in
- Ambari. (mahadev)
-
- AMBARI-1231. Replace sudo with su in the ambari setup script since ambari
- server setup is already run as root. (mahadev)
-
- AMBARI-1201. Improve Agent Registration and Heartbeat json. (Nate Cole via
- mahadev)
-
- AMBARI-1238. AmbariMetaInfoTest getServices() acceptance test failure. 
- (Siddharth Wagle via mahadev)
-
- AMBARI-1243. Remove unwanted import causing the builds to fail on linux.
- (mahadev)
-
- AMBARI-1233.  Directory permissions on httpd /var/www/cgi-bin should not be
- touched by Ambari. (mahadev)
-
- AMBARI-1170. For live status checks we should only look at the run
- directories that we get from the server (only for hadoop and its eco system)
- and not all. (mahadev)
-
- AMBARI-1250. Upgrade the posgres connector to 9.1.
- (mahadev)
-
- AMBARI-1259. Fix the host roles live status not go back to INSTALLED if it
- was in START_FAILED state. (mahadev)
-
- AMBARI-1210. Allow capacity scheduler to be attached to host role configs for
- CS configurability in the API's. (mahadev)
-
- AMBARI-1256. Host registration can fail due to mount point info not fitting
- ambari.hosts::disks_info column. (Sumit Mohanty via mahadev)
-
- AMBARI-1266. Agent checks packages as part of host check but doesn't tell
- which ones are needed or conflicting. (mahadev)
-
- AMBARI-1291. Incorrect directory for MySQL component on SLES-11.1sp1.
- (mahadev)
-
- AMBARI-1301. Live status checks dont get triggered on server restart.
- (mahadev)
-
- AMBARI-1285. Some host Ganglia metrics may be missing in some cases. (tbeerbower)
-
- AMBARI-1310. Get rid of mvn warnings. (Arun Kumar via mahadev)
-
- AMBARI-1314. Hostname test is failing in some environments. (Nate Cole via
- mahadev) 
-
- AMBARI-1330. Cluster missing hosts after successful install and restart.
- (mahadev)
-
- AMBARI-1358. Clean up alert messages. (Yusaku Sako via mahadev)
-
- AMBARI-1432. Ambari Agent registration hangs due to Acceptor bug in Jetty for
- not reading through accepted connections. (mahadev)
-
- AMBARI-1434. Change state to installed from start_failed if there is any
- issue in starting a host component. (mahadev)
-
- AMBARI-1476. Change webhcat-env.sh to export HADOOP_HOME
- (mahadev)
-
- AMBARI-1486. Fix TestHostName to take care of issues when gethostname and
- getfqdn do not match. (mahadev)
-
- AMBARI-1495. Out of Memory Issues on Ambari Server when server is running on
- single core. (mahadev)
-
- AMBARI-1487. Fix alerts at host level if MapReduce is not selected not to
- alert for tasktrackers not running. (mahadev)
-
- AMBARI-1488. Nagios script causes unwanted Datanode logs. (mahadev)
-
- AMBARI-1497. Fix start up option for ambari-server where there is a missing
- space. (mahadev)
-
- AMBARI-1498. Hive service check fails on secure HDP cluster. (Siddharth Wagle
- via mahadev)
-
- AMBARI-1299. Bootstrap can hang indefinitely. (mahadev)
-
- AMBARI-1547. Fix ambari agent test cases that are failing due to missing
- directory. (mahadev)
-
- AMBARI-1617. Host check is broken because of changing the serialization from
- jackson to gson. (mahadev)
-
-AMBARI-1.2.0 branch:
-
- INCOMPATIBLE CHANGES
- 
- NEW FEATURES
-
- AMBARI-1108. PUT call to change the state on host_components collection
- returns 200 (no op), even though GET with the same predicate returns a number
- of host_components. (Tom Beerbower via mahadev)
-
- AMBARI-1114. BootStrap fails but the api says thats its done and exit status
- is 0. (Nate Cole via mahadev)
-
- AMBARI-1136. Add gsInstaller resource provider. (Tom Beerbower via mahadev)
-
- AMBARI-1202. Unncessary use of xml tree python library in ambari-server
- setup. Its not being used. (Siddharth Wagle via mahadev)
-
- AMBARI-1769. Python REST client to invoke REST calls. (Subin M via mahadev)
-
- IMPROVEMENTS
-
- BUG FIXES
-
- AMBARI-2159. After upgrading ambari from 1.2.2.5 to 1.2.3.6 the server
- throws 500 error when starting/stopping any service. part-II (smohanty)
-
- AMBARI-2057. Gmond left in init after install, part-II. (smohanty)
-
- AMBARI-1613.  ConfigurationResourceProvider doesn't properly handle OR
-               predicate. (jspeidel) 
-
- AMBARI-1179. ambari-web does not compile due to less-brunch package update.
- (yusaku)
-
- AMBARI-1126. Change SUSE lzo dependency to only lzo-devel. (nate cole via
- mahadev)
-
-AMBARI-666 branch:
-
-  INCOMPATIBLE CHANGES
-
-  NEW FEATURES
-
-  AMBARI-1147. Handling Hive/HCat/WebHCat configuration parameters with
-  Ambari Web. (yusaku)
-
-  AMBARI-946. Support retrieving information for multiple requests.
-  (hitesh via mahadev)
-
-  AMBARI-1065. Provide Datanode decommission & recommission capability in UI.
-  (Srimanth Gunturi via mahadev)
-
-  AMBARI-985. Support OR in API query. (Tom Beerbower via mahadev)
-
-  AMBARI-1029. Add api support for updating multiple host_component resources
-  for multiple hosts in a single request. (John Speidel via mahadev)
-
-  AMBARI-1018. Add API support for creating multiple sub-resources to
-  multiple resources in a single request. (John Speidel via mahadev)
-
-  AMBARI-950. Provide API support for 'OR' predicate. (John Speidel via
-  mahadev)
-
-  AMBARI-935. Provide API support for updates of multiple resources in a
-  single request. (John Speidel via mahadev)
-
-  AMBARI-926. Provide API support for asynchronous requests.
-  (John Speidel via mahadev)
-
-  AMBARI-1054. Implement retrying of bootstrap on confirm host page.
-  (Jaimin Jetly via yusaku)
-
-  AMBARI-1048. Integrate slave configuration parameters with respective
-  service on step7 of installer wizard. (Jaimin Jetly via yusaku)
-
-  AMBARI-1031. Check for host registration at step3 of installer wizard 
-  and retrieve information for RAM and no. of cores. (Jaimin Jetly via
-  yusaku)
-
-  AMBARI-1022. Integrate Heatmap UI to backend API. (Srimanth
-  Gunturi via yusaku)
-
-  AMBARI-1015. Create HBase summary section in Dashboard & Service
-  pages. (Srimanth Gunturi via yusaku)
-
-  AMBARI-1014. Hook service summary sections in service pages to API.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1008. Populate dashboard>MapReduce section with API data.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1006. Populate dashboard>HDFS section with API data.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1004. Allow properties entered in custom config
-  (ex: hdfs-site.xml) to override existing or create new properties.
-  (Jaimin Jetly via yusaku)
-
-  AMBARI-1002. Integrate Installer with config APIs. (Jaimin Jetly
-  via yusaku)
-
-  AMBARI-989. Show task logs for each host in the Deploy step of the
-  wizard. (yusaku)
-
-  AMBARI-976.  Hook HDFS/MapReduce/HBase/Host graphs to backend API
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-964. Implement summary page of installer wizard. (Jaimin Jetly
-  via yusaku)
-
-  AMBARI-974. Decommissioning of datanodes. (Jitendra Pandey via mahadev)
-
-  AMBARI-975. Fix support for cascading updates to configs. (Hitesh Shah
-  via mahadev)
-
-  AMBARI-971. Add api support for creating multiple resources in a single
-  request. (John Speidel via mahadev)
-
-  AMBARI-970. Add additional Ganglia metrics and JMX properties. (Tom
-  Beerbower via mahadev)
-
-  AMBARI-967. Enhance predicate comparison. (Tom Beerbower via mahadev)
-
-  AMBARI-954. Support installation of Ganglia master and slaves via
-  Ambari Web. (yusaku)
-
-  AMBARI-980. Allow installation of various service components. (yusaku)
-
-  AMBARI-949. Provide metric graphs for individual hosts. (Srimanth
-  Gunturi via yusaku)
-
-  AMBARI-948. Invoke service starts after services are installed in the
-  wizard. (yusaku)
-
-  AMBARI-942. Integrate Install and Start APIs with the installer wizard.
-  (Jaimin Jetly via yusaku)
-
-  AMBARI-936. Provide HBase service specific graphs. (Srimanth Gunturi
-  via yusaku)
-
-  AMBARI-933. Provide service-specific alerts in the service summary
-  pages. (Srimanth Gunturi via yusaku)
-
-  AMBARI-938. Hardcode service name and client component mapping while
-  awaiting meta data integration. (hitesh)
-
-  AMBARI-927. Provide metrics graphs on the MapReduce services page. 
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-916. Provide metrics graphs in HDFS services page. (Srimanth
-  Gunturi via yusaku)
-
-  AMBARI-908. Add ui option to either create a Postgres database for Hive
-  and Oozie or choose existing database. (Jaimin Jetly via yusaku)
-
-  AMBARI-915. Implement App Browser for displaying and navigating Pig/Hive
-  workflows. (yusaku)
-
-  AMBARI-907. Add support for getting multiple objects in controller.
-  (hitesh)
-
-  AMBARI-906. Util to extract hosts for various components. (jitendra)
-
-  AMBARI-903. Various fixes for config handling integration. (Hitesh Shah via 
-  mahadev)
-
-  AMBARI-900. Add configuration mapping support. (Nate Cole via mahadev)
-
-  AMBARI-895. Need name consistency for metrics. (Tom Beerbower via mahadev)
-
-  AMBARI-893. provide api support for temporal queries. (John Speidel via 
-  mahadev)
-
-  AMBARI-897. Operations request object and skeleton management methods.
-  (jitendra)
-
-  AMBARI-894. TestHeartBeatMonitor fails intermittently. (jitendra)
-
-  AMBARI-892. Add puppet executor at the agent to be able to run various
-  commands from the server. (mahadev)
-
-  AMBARI-887. Ability to save configuration. (Nate Cole via mahadev)
-
-  AMBARI-877. Refactor resource provider implementation for changes to
-  management interface. (Tom Beerbower via mahadev)
-
-  AMBARI-876. Put metrics under metrics category. (Tom Beerbower via 
-  mahadev)
-
-  AMBARI-890. Add client library option to step6 (Assign slaves) of
-  installer wizard. Also add indicator and popovers for hosts with
-  master component. (Jaimin Jetly via yusaku)  
-
-  AMBARI-889. Provide cluster metric graphs on Ambari main dashboard.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-886. Support filters in controller get* apis. (hitesh)
-
-  AMBARI-880. Implement Review Page (Step 8) for the Ambari Installer
-  Wizard. (Jaimin Jetly via yusaku)
-
-  AMBARI-872. Hookup Nagios alerts section in Ambari UI to backend
-  server. (Srimanth Gunturi via yusaku)
-
-  AMBARI-871. Integrate basic set of rest APIs with ambari-web
-  installer wizardi. (Jaimin Jetly via yusaku)
-
-  AMBARI-884. Implement Dashboard/Service summary. (yusaku)
-
-  AMBARI-882. Group-based DataNode/TaskTracker/RegionServer overrides.
-  (yusaku)
-
-  AMBARI-881. Implement Add Hosts Wizard. (yusaku)
-
-  AMBARI-869. Util to deserialize ExecutionCommand. (jitendra)
-
-  AMBARI-874. Fix hostinfo reporting at the server and add a unit test for
-  deserilization for the host information from agent. (mahadev)
-
-  AMBARI-873. Support for multiple objects' updates in controller. (hitesh)
-
-  AMBARI-870. Support metric types other than string (Tom Beerbower via
-  mahadev)
-
-  AMBARI-868. Clean up site.pp generation on the agent and remove the imports
-  in the sample site.pp. (mahadev)
-
-  AMBARI-862. API query against /clusters doesn't return any data.
-  (John Speidel via mahadev)
-
-  AMBARI-866. Add ORM layer for the FSM's in the server. (mahadev)
-
-  AMBARI-853. Add more complete JMX metrics. (Tom Beerbower via mahadev)
-
-  AMBARI-852. Improve REST API functionality regarding query and partial
-  response (John Speidel via mahadev)
-
-  AMBARI-865. Add unit test for action queue on the agent. (mahadev)
-
-  AMBARI-851. Hook up Ganglia property provider. (Tom Beerbower via mahadev)
-
-  AMBARI-863. Fix mvn tests to be able to run the python tests cleanly.
-  (mahadev)
-
-  AMBARI-849. Stage planner implementation. (jitendra)
-
-  AMBARI-860. Remove code that adds fake stages for testing. (jitendra)
-
-  AMBARI-856. Add cluster heatmap. (yusaku)
-
-  AMBARI-855. Create the skeleton for a custom data adapter in Ambari Web.
-  (yusaku)
-
-  AMBARI-854. Serve ambari-web from jetty. (Jaimin Jely via yusaku)
- 
-  AMBARI-850. Flatten ExecutionCommand structure. (jitendra)
-  
-  AMBARI-848. Various tests for FSM and Controller impl. (hitesh)
-
-  AMBARI-847. Run pyunit tests from maven test target and also autogenerated
-  openssl passphrase. (mahadev)
-
-  AMBARI-845. Fix NPE in the server to be able to run the server api's.
-  (mahadev)
-
-  AMBARI-844. Mock JMX provider for manual tests. (Tom Beerbower via mahadev)
-
-  AMBARI-841. Fix comparison predicates in case where resource does not have
-  property value. (Tom Beerbower via mahadev)
-
-  AMBARI-833. Add missing Path annotation to rest services for
-  put/post/delete. (John Speidel via mahadev)
-
-  AMBARI-838. HACK: Add a thread in server to inject requests for testing.
-  (Jitendra via mahadev)
-
-  AMBARI-835. Update JMXPropertyProvider. (Tom Beerbower via hitesh)
-
-  AMBARI-832. Merge ambari-api with ambari-server (mahadev)
-
-  AMBARI-822. Implement an agent simulator for unit testing. (jitendra)
-  
-  AMBARI-829. Add unit tests for ResourceProviderImpl. (Tom Beerbower via
-  mahadev)
-
-  AMBARI-831. Move manifest generation into the ambari agent directory.
-  (mahadev)
-
-  AMBARI-828. Manifest generation for various actions from the server.
-  (mahadev)
-
-  AMBARI-827. Add clusterName to the status of the commands run by the agent.
-  (mahadev)
-
-  AMBARI-824. Provide basic management functionality (create/update) in the
-  rest api (John Speidel via mahadev)
-
-  AMBARI-826. Bug in processing command reports. (jitendra)
-
-  AMBARI-825. Controller layer implementation part 3. (hitesh)
-
-  AMBARI-823. Fix security filter on the server agent ports and remove
-  duplication on servlet contexts for certs signing. (mahadev)
-
-  AMBARI-821. Implement basic service state update and verify flow to
-  ActionManager. (hitesh)
-
-  AMBARI-812. In API , improve partial response support to drill down n levels
-  (John Spiedel)
-
-  AMBARI-791. Add unit tests and java docs for SPI code. (Tom Beerbower)
-
-  AMBARI-820. Remove JAXB dependencies in Server Agent protocol and move to
-  POJO based jackson serializer. (mahadev)
-
-  AMBARI-819. Management controller implemenation work. (hitesh)
-
-  AMBARI-811. Bug fix in jaxb serialization for maps. (jitendra)
-
-  AMBARI-810. Controller layer implementation part 1. (hitesh)
-
-  AMBARI-807. Fix Action scheduler tests because of fsm interface changes.
-  (jitendra)
-
-  AMBARI-806. Remove State object as configs/stack version/running state are
-  handled as standalone entities. (hitesh)
-
-  AMBARI-805. Add requestId tracking objects for management spi. (hitesh)
-
-  AMBARI-803. FSM initial refactoring for eventual live/desired objects. (hitesh)
-
-  AMBARI-800. Hack to add a stage for testing in in-memory db. (jitendra)
-  
-  AMBARI-801. Fix heartbeat message from the agent which is causing NPE at the
-  server. (mahadev)
-
-  AMBARI-778. Ensure data flows across all steps in installer wizard.
-  (Jaimin Jetly via yusaku)
-
-  AMBARI-799. Prototype for management spi part 3. (hitesh)
-
-  AMBARI-797. Prototype for management spi interface continued. (hitesh)
-
-  AMBARI-795. Fix failing tests for AgentResource and BootStrap. (mahadev)
-
-  AMBARI-793. Make MapReduce, Nagios, and Ganglia optional during cluster
-  install. (yusaku)
-
-  AMBARI-794. Add log4j properties for logging at the server. (mahadev)
-
-  AMBARI-790. OK in registration response. (jitendra)
-
-  AMBARI-787. Registration throws HostNotFoundException for new hosts. (jitendra)
-  
-  AMBARI-788. Fix server and agent startup for end to end testing. (mahadev)
-
-  AMBARI-785. Action response unit test. (jitendra)
-
-  AMBARI-783. Fix guice injection in the server. (mahadev)
-
-  AMBARI-784. Add Resource download API on the server. (mahadev)
-
-  AMBARI-781. Registration unit test. (jitendra)
-
-  AMBARI-754. Heartbeat handler: Registration response should query component 
-  status. (jitendra)
-
-  AMBARI-755. Heartbeat handler: Update state as reported in heartbeat. 
-  (jitendra)
-
-  AMBARI-756. Heartbeat handler: Handle heartbeat timeout. (jitendra)
-
-  AMBARI-767. Add bootstrap script to ssh in parallel and setup agents on a
-  list of hosts. (mahadev)
-
-  AMBARI-764. Integrate REST API (Tom Beerbower via mahadev)
-
-  AMBARI-762. Implement Confirm Hosts page for Ambari installer
-  (Jaimin Jetly via yusaku)
-
-  AMBARI-763. Implement Installer Step 6 (Assign Slaves). (yusaku)
-
-  AMBARI-760. Fix injection in data access objects to use guice provider.
-  (mahadev)
-
-  AMBARI-759. Add puppet scripts to the agent for configuring/installing
-  various services and add security aspects to api's and server/agent.
-  (mahadev)
-
-  AMBARI-749. Complete Java side implementation of bootstrapping agent hosts.
-  (mahadev)
-
-  AMBARI-757. Implement Installer Step 4 (Select Services). (yusaku)
-
-  AMBARI-751. Re-structure servicecomponenthost fsm layout. (hitesh)
-
-  AMBARI-732. Action scheduler unit tests. (jitendra)
-
-  AMBARI-739. Cluster fsm implementation. (hitesh)
-
-  AMBARI-738. s/Node/Host/g. (hitesh)
-
-  AMBARI-737. ServiceComponentNode FSM implementation. (hitesh)
-
-  AMBARI-722. Action scheduler implementation. (jitendra)
-  
-  AMBARI-733. Add Jersey Resource for BootStrapping and JAXB elements for API
-  entities. (mahadev)
-
-  AMBARI-730. Add unit tests for jersey apis on the server. (mahadev)
-
-  AMBARI-725. Add commandstatus/result/error objects into the rest API between
-  server and agent. (mahadev)
-
-  AMBARI-723. Implement Installer Welcome page and Install Options page
- (Jaimin Jetly via yusaku)
-
-  AMBARI-726. ORM-based data access layer for new design (Jaimin Jetly via hitesh)
-
-  AMBARI-728. Initial work on ServiceComponentNode FSM. (hitesh)
-
-  AMBARI-724. Add tabs, dynamic form generation, validation errors, and info
-  popovers for Customize Services page in Installer (yusaku) 
-
-  AMBARI-714. Job FSM Impl and tests. (hitesh)
-
-  AMBARI-721. Remove Hardwareprofile class since its not needed anymore.
-  (mahadev)
-
-  AMBARI-720. Tweak basic styles for Installer. (yusaku)
-
-  AMBARI-719. Enable Responsive Design. (yusaku)
-
-  AMBARI-716. Add back TestNodeImpl and fix memory types and disk info
-  serialization. (mahadev)
-
-  AMBARI-717. Starter implementation for Installer Customize Services page.
-  Stylize top nav and implement static footer.  Stylize login page. (yusaku)
-
-  AMBARI-711. Create utility functions related to localStorage for first two
-  steps: cluster name and Install options. Also develop view logic with
-  preliminary validations for these two steps. (Jaimin Jetly via yusaku)
-
-  AMBARI-715. Integrate domain objects and Rest serialized objects. (mahadev)
-
-  AMBARI-713. Initial work on Job FSM. (hitesh)
-
-  AMBARI-712. Action manager skeleton. (jitendra)
-
-  AMBARI-710. Basic registration and heartbeat protocol implementation between
-  the server and the agent. (mahadev)
-
-  AMBARI-709. Getting hardware info on disks/cpu/others using facter and using
-  it during registeration. (mahadev)
-
-  AMBARI-707. More work on Node FSM and additional tests/cleanup. (hitesh)
-
-  AMBARI-706. Basic tests for Node FSM. (hitesh)
-
-  AMBARI-705. Initial work on Node FSM. (hitesh)
-
-  AMBARI-703. Heartbeat handler classes. (jitendra)
-
-  AMBARI-702. Add skeleton for Ambari agent that talks to the server and
-  collects information for host. (mahadev)
-
-  AMBARI-696. Add interface for ActionManager to access live state. (hitesh)
-
-  AMBARI-698. Add a simple server and artifact generation to run a server with
-  a simple api check. (mahadev)
-
-  AMBARI-697. Ambari Web (browser-based UI) skeleton. (Jaimin Jetly and yusaku)
-
-  AMBARI-695. More basic class restructuring for new design. (hitesh)
-
-  AMBARI-694. Class to encapsulate stage. (jitendra)
-
-  AMBARI-691. More basic classes for new design. (hitesh)
-
-  AMBARI-693. Classes for request objects. (jitendra)
-
-  AMBARI-685. Basic classes. (hitesh via jitendra)
-
-  AMBARI-676. Seperate directory for ambari-server. (jitendra)
-
-  IMPROVEMENTS
-
-  AMBARI-1159. Check the log/run dir locations to make sure its an abs path.
-  (yusaku)
-
-  AMBARI-1156. Dashboard > HDFS pie chart should hover with details. (yusaku)
-
-  AMBARI-1154. The check boxes to check/uncheck one of the members in a multi
-  artifact graphs is not very readable. It should be more apparent on which
-  one the user clicked on. (yusaku)
-
-  AMBARI-1106. User-specified custom configs (such as hdfs-site.xml overrides)
-  should be persisted to maintain what the user specified.
-  (Jaimin Jetly via yusaku)
-
-  AMBARI-1103. Need to be able to reliably recover from the case when the browser
-  is closed during deploy (Step 8 post submission, Step 9) of the wizard.
-  (Arun Kandregula via yusaku)
-
-  AMBARI-1099. Hive Service Summary needs to show service components better.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1097.  Restrict user to proceed ahead of step 7 (Service configuration)
-  when properties specified in custom-box are already exposed on the page.
-  (Jaimin Jetly via yusaku)
-
-  AMBARI-1102. Error handling when errors are encountered during preparation
-  for deploy. (Arun Kandregula via yusaku)
-
-  AMBARI-1096. Create heatmap legend entries for missing data/invalid hosts.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1078. Improve graph message when data is not available.
-  (Srimanth Gunturi via yusaku)
- 
-  AMBARI-1146. Exclude hosts and include hosts config parameters need 
-  clarification. (yusaku)
-  
-  AMBARI-1074. CPU Usage chart needs better idle time display. (Srimanth
-  Gunturi via yusaku)
-
-  AMBARI-1072. Change text on alerts "about XX hours ago". (Srimanth Gunturi
-  via yusaku)
-
-  AMBARI-1145. Cluster Management refactoring. (yusaku)
-
-  AMBARI-984. Add support for exposing filesystem type for mount points
-  at host level. (hitesh via mahadev)
-
-  AMBARI-973. Ensure zookeeper service check run after zookeeper start.
-  (hitesh via mahadev)
-
-  AMBARI-965. Stop events should be handled at all valid points for safe
-  recovery. (hitesh via mahadev)
-
-  AMBARI-959. Handle passing repo information from server to agent.
-  (hitesh via mahadev)
-
-  AMBARI-951. Integrate meta data to be able to run service checks after
-  a start of a service. (hitesh via mahadev)
-
-  AMBARI-932. Add initial hooks to trigger smoke tests on service starts.
-  (hitesh via mahadev)
-
-  AMBARI-924. Rename job to action. (hitesh via mahadev)
-
-  AMBARI-922. Use stack name and version in fsm layer. (hitesh via mahadev)
-
-  AMBARI-982. Add ability to set rack info for a host.
-  (Nate Cole via mahadev)
-
-  AMBARI-981. Add configuration to host_component request.
-  (Nate Cole via mahadev)
-
-  AMBARI-931. Support for /hosts end point. (Nate Cole via mahadev)
-
-  AMBARI-912. Test case for ConfigurationService.
-  (Nate Cole via mahadev)
-
-  AMBARI-1021. Ambari-agent init script needs to be aware of already
-  running/not running agent process. (Pramod Thangali via mahadev)
-
-  AMBARI-1019. Add methods to get postgres user name/password or any
-  regex validated string input from user.
-  (Pramod Thangali via mahadev)
-
-  AMBARI-1007. Add aggregate IO stats to workflow data web service
-  (Pramod Thangali via mahadev)
-
-  AMBARI-1000. Use FQDN instead of hostname when registering an agent with
-  Ambari server. (Pramod Thangali via mahadev)
-
-  AMBARI-1066. Rename Charts section to Heatmaps. (Srimanth Gunturi via
-  mahadev)
-
-  AMBARI-1056. Expose CapacityRemaining JMX metric to NAMENODE
-  ServiceComponentInfo. (Tom Beerbower via mahadev)
-
-  AMBARI-1055. Refactor SPI Request interface to remove PropertyId.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-1049. Tune Ganglia request. (Tom Beerbower via mahadev)
-
-  AMBARI-1037. Implement an efficient way to provide Ganglia data for
-  heatmap and other cluster visualization tools.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-1001. Cluster level Network, Load, CPU and Memory metrics in API
-  needed for dashboard page. (Tom Beerbower via mahadev)
-
-  AMBARI-996. Expose metrics and properties for UI. (Tom Beerbower via mahadev)
-
-  AMBARI-972. Refactor resource provider implementation to move inner classes.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-943. Add Host metrics. (Tom Beerbower via mahadev)
-
-  AMBARI-929. Show HBASE_REGIONSERVER metrics. (Tom Beerbower via mahadev)
-
-  AMBARI-928. Enable end to end testing of Request and Task resources.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-925. Return RequestStatus through ClusterController.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-923. ResourceProvider changes for Request and Task resources.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-911. Implement an efficient way to provide Ganglia data for heatmap
-  and other cluster visualization tools. (Tom Beerbower via mahadev)
-
-  AMBARI-930. Map update to PUT and create to POST.
-  (John Speidel via mahadev)
-
-  AMBARI-1053. Dashboard page loads very slow due to hosts?fields=* API call
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1051. Dashboard page takes long time to load. (Srimanth Gunturi via
-  yusaku)
-
-  AMBARI-1041. Additional metrics need to be added to Heatmap UI. (Srimanth
-  Gunturi via yusaku)
-
-  AMBARI-1040. Cluster heatmap: green should always mean "good". (Srimanth
-  Gunturi via yusaku)
-
-  AMBARI-1039. Improve Nagios alerts time display. (Srimanth Gunturi via yusaku)
-
-  AMBARI-1036. Service Info/Quick Links do not display external hostnames.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1035. Aggregate creation of multiple services and assignment of host
-  to cluster. (Jaimin Jetly via yusaku)
-
-  AMBARI-1034. Metric Charts - display local time rather than UTC.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1033. Nagios and Ganglia links should use public host names in URLs.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1030. Metrics links in web ui should link to Ganglia UI. (Srimanth
-  Gunturi via yusaku)
-
-  AMBARI-1025. Display total install and start services time on summary page
-  and polish summary page ui. (Jaimin Jetly via yusaku)
-
-  AMBARI-1023. Dashboard page should handle API sending JSON as strings and
-  object. (Srimanth Gunturi via yusaku)
-
-  AMBARI-1011. Create 2 missing HDFS service graphs. (Srimanth Gunturi via
-  yusaku)
-
-  AMBARI-1003. Nagios sections should use backend API to populate. (Srimanth
-  Gunturi via yusaku)
-
-  AMBARI-1062. Convert Apache license header comment style in Handlebars files
-  to Handlebars comments rather than JavaScript comments. (yusaku)
-
-  AMBARI-1061. Data loading refactoring for cluster management. (yusaku)
-
-  AMBARI-1060. Data loading for App Browser. (yusaku)
-
-  AMBARI-993. Hook up login with server authentication. (yusaku)
-
-  AMBARI-1059. Refactor cluster management. (yusaku)
-
-  AMBARI-1058. Implement data loading. (yusaku)
-
-  AMBARI-956. On unavailability of non-master components, host with least
-  number of master components should install all slave and client components. 
-  (Jaimin Jetly via yusaku)
- 
-  AMBARI-990. Refactor App Browser. (yusaku)
-
-  AMBARI-979. More refactoring of App Browser code. (yusaku)
-
-  AMBARI-947. Make it easier to test Deploy (Install, Start + Test) step
-  of the wizard. (yusaku)
-
-  AMBARI-978. Refactor App Browser code. (yusaku)
-
-  AMBARI-977. Refactor Wizard and Cluster Management code. (yusaku)
-
-  AMBARI-941. More refactoring of Wizards in Ambari Web. (yusaku)
-
-  AMBARI-919. Partial refactoring and consolidation of code for various
-  wizards. (yusaku)
-
-  AMBARI-918. Update styles in Cluster Management. (yusaku)
-
-  AMBARI-917. Update layout and flow for App Browser. (yusaku)
-
-  AMBARI-888. Add more tests for controller implementation. (hitesh)
-
-  AMBARI-891. Initial work to refactor the Wizards in Ambari Web. (yusaku)
-
-  AMBARI-883. Improve user interactions on Confirm Hosts page of the
-  Installer. (yusaku)
-
-  AMBARI-859. Tighten up the layout for the Install page of the Installer.
-  (yusaku)
-
-  AMBARI-857. Refactor Install Options page for the Install Wizard. (yusaku)
-
-  OPTIMIZATIONS
-
-  BUG FIXES
-
-  AMBARI-1628. Tasktracker remains in STARTING state preventing Ambari 
-  to display proper status. (Sumit Mohanty via swagle)
-
-  AMBARI-1160. Cannot add a hostname that has a number next to . in it.
-  (yusaku)
-
-  AMBARI-1139. Disable Misc section in Customize Services page of the Install
-  Wizard. (Srimanth Gunturi via yusaku) 
-
-  AMBARI-1158. Fiters are not working correctly on Hosts page. (yusaku)
-
-  AMBARI-1157. Host component operation causes lags in status/action pulldown
-  update. (yusaku)
-
-  AMBARI-1144. Cannot save changes to ZooKeeper configuration.
-  (Arun Kandregula via yusaku)
-
-  AMBARI-1155. Change "Save and apply changes" button on configs section to
-  "Save". (yusaku)
-
-  AMBARI-1153. Host jams in status 'Preparing' if host name is wrong.
-  (Arun Kandregula via yusaku)
-
-  AMBARI-1132. Stopping service doesn't cause blinking status until refresh.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1143. tmpfs filesystem being added to the list in the dir used by
-  Ambari. (Arun Kandregula via yusaku) 
-
-  AMBARI-1142. On Notification Popup, clicking "go to nagios UI" doesn't
-  load nagios UI. (Arun Kandregula via yusaku)
-
-  AMBARI-1125. Graphs "degrade" over time. (Srimanth Gunturi via yusaku)
-
-  AMBARI-1124. Boxes on services page need min height or something to keep
-  it from visually cutting off info. (Srimanth Gunturi via yusaku)
-
-  AMBARI-1123. Ambari heatmaps and host information shows infinity for disk
-  space used. (Srimanth Gunturi via yusaku)
-
-  AMBARI-1141. In some cases, clicking "Register and Confirm" button does
-  not do anything. (Arun Kandregula via yusaku)
- 
-  AMBARI-1140. Resuming deploy for Installer/Add Hosts does not work if the
-  browser is shut down during the start phase of deploy.
-  (Arun Kandregula via yusaku)
-  
-  AMBARI-1120. Key spikes in HDFS IO missing from IO summary graphs.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1119. Service Summary pages no longer show service-specific info.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1118. Dashboard > HDFS title's free capacity doesn't match summary.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1115. Host component live status is broken. (Srimanth Gunturi via
-  yusaku)
-
-  AMBARI-1113. Install Wizard: Confirm host stuck at Preparing stage.
-  (Arun Kandregula via yusaku)
-
-  AMBARI-1112. Add hosts fails second time around. (Srimanth Gunturi via
-  yusaku)
-
-  AMBARI-1111. Install wizard step 9 gets stuck at 0% and other issues on
-  page refresh. (Srimanth Gunturi via yusaku)
-
-  AMBARI-1110. After clicking the deploy button on the Add Hosts wizard, the
-  user is always taken to the Installer Wizard Step 8 upon login.
-  (Arun Kandregula via yusaku)
-
-  AMBARI-1152. Add Hosts wizard - Retry button does not trigger call to
-  backend. (yusaku)
-
-  AMBARI-1104. Webhcat configuration not setting templeton-libjars.
-  (Jaimin Jetly via yusaku)
-
-  AMBARI-1151. Reconfigure fails silently; it's not firing any API calls due
-  to a JS error. (yusaku)
-
-  AMBARI-1098. Switching services does not update various UI elements.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1150. Installer Wizard - Retry feature in Deploy step (Step 9) is
-  broken. (yusaku)
-
-  AMBARI-1092. dashboard > Summary > capacity pie chart keeps changing colors.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1091. 2 parallel requests for service information resulting in JS
-  exception. (Srimanth Gunturi via yusaku)
-
-  AMBARI-1090. Restrict user to apply service configuration when custom box
-  properties are already exposed on the management config page.
-  (Jaimin Jetly via yusaku)
-
-  AMBARI-1149. HIVE_METASTORE needs to be started as a Hive component.
-  (yusaku)
-
-  AMBARI-1088. HDFS capacity chart numbers are incorrect. (Srimanth Gunturi
-  via yusaku)
-
-  AMBARI-1084. Heatmap displays NaN. (Srimanth Gunturi via yusaku)
-
-  AMBARI-1081. HDFS disk capacity on dashboard is seen as negative number.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1148. Fix incorrect labels for configuration parameters. (yusaku)
-
-  AMBARI-1080. Host disk & memory graphs have incorrect values.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1077. The value for dead nodes is not getting populated on UI.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1073. Remove cluster name entry from top titlebar. (Srimanth Gunturi
-  via yusaku)
-
-  AMBARI-1071. Nagios alerts not updating in UI. (Srimanth Gunturi via yusaku)
-
-  AMBARI-1063. Workflow Web Service potentially leaks DB connections upon
-  exceptions. (yusaku)
-
-  AMBARI-962. Update of host components fail when service name is not passed
-  in. (hitesh via mahadev)
-
-  AMBARI-945. Fix 500 errors on get resources api. (hitesh via mahadev)
-
-  AMBARI-944. Fixes for meta info layer. (hitesh via mahadev)
-
-  AMBARI-913. Fix all apis to return correctly filled status response.
-  (hitesh via mahadev)
-
-  AMBARI-999. RUBYLIB env variable expanding forever. (Pramod Thangali via
-  mahadev)
-
-  AMBARI-1069. HDFS Disk Capacity in HDFS Service Summary is totally off.
-  (Srimanth Gunturi via mahadev)
-
-  AMBARI-1068. Dashboard cluster level graphs showing only 45 minutes of data.
-  (Srimanth Gunturi via mahadev)
-
-  AMBARI-1067. Service > MapReduce map slots reserved / occupied are backwards.
-  (Srimanth Gunturi via mahadev)
-
-  AMBARI-1057. Can't reset ambari-server due to fail drop/create database DDLs.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-1052. UnsupportedPropertyException thrown from update.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-1050. Host metric values coming in with 0 values.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-1047. Create Configuration API call is throwing 500:
-  UnsupportedPropertyException. (Tom Beerbower via mahadev)
-
-  AMBARI-1044. API is not returning Ganglia metrics for one of the hosts
-  in the cluster. (Tom Beerbower via mahadev)
-
-  AMBARI-1043. Updates with queries that contain non primary key fields
-  may update resources that don't satisfy the query.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-1038. Ganglia setup does not allow for subsequent changes to the
-  cluster. (Tom Beerbower via mahadev)
-
-  AMBARI-1027. Fix missing block metrics for NAMENODE.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-1026. Resolve overlap between JMX and Ganglia metrics.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-1020. Start time, end time and resolution not set correctly for
-  rrd.py call in Ganglia property. (Tom Beerbower via mahadev)
-
-  AMBARI-1016. Initial API calls after Ambari Web install resulted in 500
-  (ArrayIndexOutOfBoundsException); Ambari Web stuck at "Loading..."
-  (Tom Beerbower via mahadev)
-
-  AMBARI-1012. Fix race condition in DefaultProviderModule.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-1010. Fix extra comma in rrd.py output. (Tom Beerbower via mahadev)
-
-  AMBARI-1005. No Ganglia/JMX metrics data are coming through.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-997. Fix HBASE JMX properties. (Tom Beerbower via mahadev)
-
-  AMBARI-994. Host metrics API servlet not filtering on given fields.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-988. Update resource drops property values.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-969. GET on temporal data throws 500. (Tom Beerbower via mahadev)
-
-  AMBARI-968. Task resources not returned under request resources.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-961. Sub-resources and metrics through API are broken.
-  (Tom Beerbower via mahadev)
-
-  AMBARI-1046. Heatmap with no numbers on the hover. (Srimanth Gunturi via
-  yusaku)
-
-  AMBARI-1045. Service summary sections have incorrect values displayed.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1042. Heatmap UI fluctuates between white and green colors
-  intermittently. (Srimanth Gunturi via yusaku)
-
-  AMBARI-1032. Host hover in cluster heatmap showing mock data. (Srimanth
-  Gunturi via yusaku)
- 
-  AMBARI-1028. MapReduce & HDFS summaries should use ServiceComponentInfo
-  values. (Srimanth Gunturi via yusaku)
-
-  AMBARI-1017. Alerts not showing up in Ambari UI due to model refactoring.
-  (Srimanth Gunturi via yusaku)
-
-  AMBARI-1013. Host metrics charts should use live data. (Srimanth Gunturi
-  via yusaku)
-
-  AMBARI-1009. Cluster level graphs need to use API for data. (Srimanth
-  Gunturi via yusaku)
-
-  AMBARI-1064. App Browser fixes. (yusaku)
-
-  AMBARI-995. Deploy logs not shown for failed tasks. (yusaku)
-
-  AMBARI-992. Logout does not clean application state properly. (yusaku)
-
-  AMBARI-957. Adding a host whose hostname is the same as the one the user 
-  is accessing Ambari Web with breaks the Installer. (yusaku)
-
-  AMBARI-953. Fix navigation issues in installer wizard due to regression.
-  (Jaimin Jetly via yusaku)
-
-  AMBARI-899. Use slf4j-api 1.5.5 rather than 1.6.6. (yusaku)
-
-  AMBARI-902. Fix ZooKeeper badge allocation logic for single host and 
-  muliple hosts installation in step5 installer wizard. (Jaimin Jetly via
-  yusaku)
-
-  AMBARI-896. Resolve all navigation related issues for Step6 (Slave and
-  Client component) of installer wizard. (Jaimin Jetly via yusaku)
-
-  AMBARI-914. Fix issues related to Slave Component Group in Installer.
-  (yusaku)
-
-  AMBARI-909. Pass correct cluster info to Action Manager. (hitesh)
-
-  AMBARI-904. Ensure state changes only happen after actionmanager persists
-  actions. (hitesh)
-
-  AMBARI-905. Fix puppet site creation with flattening of execution commands
-  send from the server. (mahadev)
-
-  AMBARI-885. Fix miscellaneous issues related to Ambari Web. (yusaku)
-
-  AMBARI-879. Installer skips Confirm Hosts page of the wizard when testing
-  locally without Ambari Server. (yusaku)
-
-  AMBARI-878. Various tests for FSM, controller and state objects. (hitesh)
-
-  AMBARI-858. Installer -> Select Services page: warning popups are no longer
-  appearing. (yusaku)
-
-  AMBARI-846. Select Masters Page: make ZooKeeper addition/removal UI more
-  organized. (Jaimin Jetly via yusaku)
-
-  AMBARI-840. Hitting browser refresh should not clear present step data that 
-  had already been persisted to local DB. (Jaimin Jetly via yusaku)
-
-  AMBARI-843. Fix more null pointers for partial request objects. (hitesh)
-
-  AMBARI-842. Fix null point exception during adding of hosts to cluster. (hitesh)
-
-  AMBARI-839. Temporary fix for server start order. (hitesh)
-
-  AMBARI-837. Fix basic injection issues for controller impl. (hitesh)
-
-  AMBARI-836. Fix generation of requestId to be unique across restarts. (hitesh)
-
-  AMBARI-834. Use RoleCommand instead of ServiceComponentHostEventType for HostAction
-  in Controller. (hitesh)
-
-  AMBARI-830. Various fixes and tests for controller implementation. (hitesh)
-
-  AMBARI-808. Handle appropriate start/stop/install/.. events at their respective
-  failed states. (hitesh)
-
-  AMBARI-798. Fix import issue due to move of Predicate class. (hitesh)
-
-  AMBARI-780. Make FSM related changes for heartbeat handler. (hitesh)
-
-  AMBARI-774. Renable and fix AgentResourceTest. (mahadev) 
-
-  AMBARI-773. Change Host FSM as per new requirements of heartbeat handler. (hitesh)
-
-  AMBARI-753. Fix broken compile as a result of re-factor of FSM layout. (hitesh)
-
-  AMBARI-752. Add missing license header to TestServiceComponentHostState. (hitesh)
-
-  AMBARI-718. Fix installer navigation. (yusaku)
-
-  AMBARI-684. Remove non-required dependencies from pom files (hitesh via jitendra)
-
-  AMBARI-680. Fix pom structure. (hitesh)
-

+ 1 - 1
ambari-project/pom.xml

@@ -231,7 +231,7 @@
       <dependency>
         <groupId>commons-codec</groupId>
         <artifactId>commons-codec</artifactId>
-        <version>1.4</version>
+        <version>1.8</version>
       </dependency>
       <dependency>
         <groupId>commons-lang</groupId>

+ 29 - 0
ambari-server/pom.xml

@@ -557,6 +557,11 @@
       <groupId>commons-httpclient</groupId>
       <artifactId>commons-httpclient</artifactId>
     </dependency>
+    <dependency>
+      <groupId>commons-net</groupId>
+      <artifactId>commons-net</artifactId>
+      <version>1.4.1</version>
+    </dependency>
     <dependency>
       <groupId>javax.servlet</groupId>
       <artifactId>servlet-api</artifactId>
@@ -638,6 +643,30 @@
       <version>3.1</version>
       <scope>test</scope>
     </dependency>
+    <dependency>
+      <groupId>org.powermock</groupId>
+      <artifactId>powermock-core</artifactId>
+      <version>1.5</version>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.powermock</groupId>
+      <artifactId>powermock-reflect</artifactId>
+      <version>1.5</version>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.powermock</groupId>
+      <artifactId>powermock-api-easymock</artifactId>
+      <version>1.5</version>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.powermock</groupId>
+      <artifactId>powermock-module-junit4</artifactId>
+      <version>1.5</version>
+      <scope>test</scope>
+    </dependency>
     <dependency>
       <groupId>org.objenesis</groupId>
       <artifactId>objenesis-tck</artifactId>

+ 9 - 1
ambari-server/sbin/ambari-server

@@ -90,8 +90,16 @@ case "$1" in
         echo -e "Setup ambari-server"
         $PYTHON /usr/sbin/ambari-server.py $@
         ;;
+  setupldap)
+        echo -e "Setting up ldap properties"
+        $PYTHON /usr/sbin/ambari-server.py $@
+        ;;
+  resetmasterkey)
+        echo -e "Reseting master key for credential store"
+        $PYTHON /usr/sbin/ambari-server.py $@
+        ;;
   *)
-        echo "Usage: /usr/sbin/ambari-server {start|stop|restart|setup|upgrade|status|upgradestack} [options]"
+        echo "Usage: /usr/sbin/ambari-server {start|stop|restart|setup|upgrade|status|upgradestack|setupldap|resetmasterkey} [options]"
         exit 1
 esac
 

+ 69 - 13
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java

@@ -18,16 +18,17 @@
 package org.apache.ambari.server.configuration;
 
 import com.google.inject.Singleton;
+import org.apache.ambari.server.AmbariException;
 import org.apache.ambari.server.orm.JPATableGenerationStrategy;
 import org.apache.ambari.server.orm.PersistenceType;
 import org.apache.ambari.server.security.ClientSecurityType;
 import org.apache.ambari.server.security.authorization.LdapServerProperties;
+import org.apache.ambari.server.security.encryption.CredentialProvider;
 import org.apache.ambari.server.utils.ShellCommandUtil;
 import org.apache.commons.io.FileUtils;
 import org.apache.commons.lang.RandomStringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
-
 import java.io.File;
 import java.io.FileNotFoundException;
 import java.io.IOException;
@@ -73,7 +74,6 @@ public class Configuration {
   public static final String SERVER_VERSION_FILE = "server.version.file";
   public static final String SERVER_VERSION_KEY = "version";
 
-
   public static final String CLIENT_SECURITY_KEY = "client.security";
   public static final String CLIENT_API_PORT_KEY = "client.api.port";
   public static final String SERVER_DB_NAME_KEY = "server.jdbc.database";
@@ -130,7 +130,6 @@ public class Configuration {
   public static final String SERVER_JDBC_RCA_DRIVER_KEY = "server.jdbc.rca.driver";
   public static final String SERVER_JDBC_RCA_URL_KEY = "server.jdbc.rca.url";
 
-
   public static final String SERVER_JDBC_GENERATE_TABLES_KEY = "server.jdbc.generateTables";
 
   public static final String JDBC_UNIT_NAME = "ambari-server";
@@ -208,14 +207,22 @@ public class Configuration {
   public static final String HIVE_METASTORE_PASSWORD_PROPERTY =
     "javax.jdo.option.ConnectionPassword";
 
+  public static final String MASTER_KEY_PERSISTED = "security.master" +
+    ".key.ispersisted";
+  public static final String MASTER_KEY_LOCATION = "security.master.key" +
+    ".location";
+  public static final String MASTER_KEY_ENV_PROP = "ambari.security.master" +
+    ".key";
+  public static final String MASTER_KEY_FILENAME_DEFAULT = "master";
+
   private static final Logger LOG = LoggerFactory.getLogger(
       Configuration.class);
 
   private Properties properties;
 
-
   private Map<String, String> configsMap;
 
+  private CredentialProvider credentialProvider = null;
 
   public Configuration() {
     this(readConfigFile());
@@ -283,6 +290,7 @@ public class Configuration {
     configsMap.put(SRVR_CRT_PASS_KEY, randStr);
 
     loadSSLParams();
+    loadCredentialProvider();
   }
 
   /**
@@ -300,6 +308,12 @@ public class Configuration {
     }
   }
 
+  private void loadCredentialProvider() {
+    if (credentialProvider == null) {
+      this.credentialProvider = new CredentialProvider(null,
+        getMasterKeyLocation(), isMasterKeyPersisted());
+    }
+  }
 
   /**
    * Find, read, and parse the configuration file.
@@ -314,7 +328,6 @@ public class Configuration {
     if (inputStream == null)
       throw new RuntimeException(CONFIG_FILE + " not found in classpath");
 
-
     // load the properties
     try {
       properties.load(inputStream);
@@ -457,8 +470,12 @@ public class Configuration {
   }
 
   public String getDatabasePassword() {
-    String filePath = properties.getProperty(SERVER_JDBC_USER_PASSWD_KEY);
-    return readPassword(filePath, SERVER_JDBC_USER_PASSWD_DEFAULT);
+    String passwdProp = properties.getProperty(SERVER_JDBC_USER_PASSWD_KEY);
+    String dbpasswd = readPasswordFromStore(passwdProp);
+    if (dbpasswd != null)
+      return dbpasswd;
+    else
+      return readPasswordFromFile(passwdProp, SERVER_JDBC_USER_PASSWD_DEFAULT);
   }
 
   public String getRcaDatabaseDriver() {
@@ -474,11 +491,15 @@ public class Configuration {
   }
 
   public String getRcaDatabasePassword() {
-    String filePath = properties.getProperty(SERVER_JDBC_RCA_USER_PASSWD_KEY);
-    return readPassword(filePath, SERVER_JDBC_RCA_USER_PASSWD_DEFAULT);
+    String passwdProp = properties.getProperty(SERVER_JDBC_RCA_USER_PASSWD_KEY);
+    String dbpasswd = readPasswordFromStore(passwdProp);
+    if (dbpasswd != null)
+      return dbpasswd;
+    else
+      return readPasswordFromFile(passwdProp, SERVER_JDBC_RCA_USER_PASSWD_DEFAULT);
   }
 
-  private String readPassword(String filePath, String defaultPassword) {
+  private String readPasswordFromFile(String filePath, String defaultPassword) {
     if (filePath == null) {
       LOG.debug("DB password file not specified - using default");
       return defaultPassword;
@@ -494,6 +515,25 @@ public class Configuration {
     }
   }
 
+  private String readPasswordFromStore(String aliasStr) {
+    String password = null;
+    loadCredentialProvider();
+    if (credentialProvider != null) {
+      char[] result = null;
+      try {
+        result = credentialProvider.getPasswordForAlias(aliasStr);
+      } catch (AmbariException e) {
+        LOG.error("Error reading from credential store.");
+        e.printStackTrace();
+      }
+      if (result != null) {
+        password = new String(result);
+      } else {
+        LOG.error("Cannot read password for alias = " + aliasStr);
+      }
+    }
+    return password;
+  }
 
   /**
    * Gets parameters of LDAP server to connect to
@@ -513,8 +553,14 @@ public class Configuration {
             LDAP_BIND_ANONYMOUSLY_DEFAULT)));
     ldapServerProperties.setManagerDn(properties.getProperty(
         LDAP_MANAGER_DN_KEY));
-    ldapServerProperties.setManagerPassword(properties.getProperty(
-        LDAP_MANAGER_PASSWORD_KEY));
+    String ldapPasswd = readPasswordFromStore(properties
+      .getProperty(LDAP_MANAGER_PASSWORD_KEY));
+    if (ldapPasswd != null) {
+      ldapServerProperties.setManagerPassword(ldapPasswd);
+    } else {
+      ldapServerProperties.setManagerPassword(properties.getProperty
+        (LDAP_MANAGER_PASSWORD_KEY));
+    }
     ldapServerProperties.setBaseDN(properties.getProperty
         (LDAP_BASE_DN_KEY, LDAP_BASE_DN_DEFAULT));
     ldapServerProperties.setUsernameAttribute(properties.
@@ -543,7 +589,6 @@ public class Configuration {
     return properties.getProperty(BOOTSTRAP_MASTER_HOSTNAME, defaultValue);
   }
 
-
   public int getClientApiPort() {
     return Integer.parseInt(properties.getProperty(CLIENT_API_PORT_KEY, String.valueOf(CLIENT_API_PORT_DEFAULT)));
   }
@@ -576,4 +621,15 @@ public class Configuration {
   public String getAnonymousAuditName() {
     return properties.getProperty(ANONYMOUS_AUDIT_NAME_KEY, "_anonymous");
   }
+
+  public boolean isMasterKeyPersisted() {
+    String isPersisted = properties.getProperty(MASTER_KEY_PERSISTED, "true");
+    return isPersisted.toLowerCase().equals("true");
+  }
+
+  public String getMasterKeyLocation() {
+    String defaultDir = properties.getProperty(MASTER_KEY_LOCATION,
+      properties.getProperty(SRVR_KSTR_DIR_KEY, SRVR_KSTR_DIR_DEFAULT));
+    return defaultDir + File.separator + MASTER_KEY_FILENAME_DEFAULT;
+  }
 }

+ 0 - 1
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java

@@ -425,7 +425,6 @@ public class AmbariServer {
   }
 
   public static void main(String[] args) throws Exception {
-
     Injector injector = Guice.createInjector(new ControllerModule());
     AmbariServer server = null;
     try {

+ 0 - 2
ambari-server/src/main/java/org/apache/ambari/server/security/CertificateManager.java

@@ -63,9 +63,7 @@ public class CertificateManager {
        */
   public void initRootCert() {
     LOG.info("Initialization of root certificate");
-
     boolean certExists = isCertExists();
-
     LOG.info("Certificate exists:" + certExists);
 
     if (!certExists) {

+ 144 - 0
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/AESEncryptor.java

@@ -0,0 +1,144 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.security.encryption;
+
+import java.security.InvalidAlgorithmParameterException;
+import java.security.InvalidKeyException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.security.spec.InvalidKeySpecException;
+import java.security.spec.InvalidParameterSpecException;
+import java.security.spec.KeySpec;
+import javax.crypto.Cipher;
+import javax.crypto.NoSuchPaddingException;
+import javax.crypto.SecretKey;
+import javax.crypto.SecretKeyFactory;
+import javax.crypto.spec.IvParameterSpec;
+import javax.crypto.spec.PBEKeySpec;
+import javax.crypto.spec.SecretKeySpec;
+
+public class AESEncryptor {
+  private static final int ITERATION_COUNT = 65536;
+  private static final int KEY_LENGTH = 128;
+  private Cipher ecipher;
+  private Cipher dcipher;
+  private SecretKey secret;
+  private byte[] salt = null;
+  private char[] passPhrase = null;
+
+  public AESEncryptor(String passPhrase) {
+    try {
+      this.passPhrase = passPhrase.toCharArray();
+      salt = new byte[8];
+      SecureRandom rnd = new SecureRandom();
+      rnd.nextBytes(salt);
+
+      SecretKey tmp = getKeyFromPassword(passPhrase);
+      secret = new SecretKeySpec(tmp.getEncoded(), "AES");
+
+      ecipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
+      ecipher.init(Cipher.ENCRYPT_MODE, secret);
+
+      dcipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
+      byte[] iv = ecipher.getParameters().getParameterSpec(IvParameterSpec.class).getIV();
+      dcipher.init(Cipher.DECRYPT_MODE, secret, new IvParameterSpec(iv));
+    } catch (NoSuchAlgorithmException e) {
+      e.printStackTrace();
+    } catch (NoSuchPaddingException e) {
+      e.printStackTrace();
+    } catch (InvalidKeyException e) {
+      e.printStackTrace();
+    } catch (InvalidParameterSpecException e) {
+      e.printStackTrace();
+    } catch (InvalidAlgorithmParameterException e) {
+      e.printStackTrace();
+    }
+  }
+
+  AESEncryptor(SecretKey secret) {
+    try {
+      this.secret = new SecretKeySpec (secret.getEncoded(), "AES");
+
+      ecipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
+      ecipher.init(Cipher.ENCRYPT_MODE, secret);
+
+      dcipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
+      byte[] iv = ecipher.getParameters().getParameterSpec(IvParameterSpec.class).getIV();
+      dcipher.init(Cipher.DECRYPT_MODE, secret, new IvParameterSpec(iv));
+    } catch (NoSuchAlgorithmException e) {
+      e.printStackTrace();
+    } catch (NoSuchPaddingException e) {
+      e.printStackTrace();
+    } catch (InvalidKeyException e) {
+      e.printStackTrace();
+    } catch (InvalidParameterSpecException e) {
+      e.printStackTrace();
+    } catch (InvalidAlgorithmParameterException e) {
+      e.printStackTrace();
+    }
+  }
+
+  public SecretKey getKeyFromPassword(String passPhrase) {
+    return getKeyFromPassword(passPhrase, salt);
+  }
+
+  public SecretKey getKeyFromPassword(String passPhrase, byte[] salt) {
+    SecretKeyFactory factory;
+    SecretKey key = null;
+    try {
+      factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
+      KeySpec spec = new PBEKeySpec(passPhrase.toCharArray(), salt, ITERATION_COUNT, KEY_LENGTH);
+      key = factory.generateSecret(spec);
+    } catch (NoSuchAlgorithmException e) {
+      e.printStackTrace();
+    } catch (InvalidKeySpecException e) {
+      e.printStackTrace();
+    }
+
+    return key;
+  }
+
+  public EncryptionResult encrypt(String encrypt) throws Exception {
+    byte[] bytes = encrypt.getBytes("UTF8");
+    EncryptionResult atom = encrypt(bytes);
+    return atom;
+  }
+
+  public EncryptionResult encrypt(byte[] plain) throws Exception {
+    EncryptionResult atom = new EncryptionResult(salt, ecipher.getParameters().getParameterSpec(IvParameterSpec.class).getIV(), ecipher.doFinal(plain));
+    return atom;
+  }
+
+  public String decrypt(String salt, String iv, String cipher) throws Exception {
+    byte[] decrypted = decrypt(salt.getBytes("UTF8"), iv.getBytes("UTF8"), cipher.getBytes("UTF8"));
+    return new String(decrypted, "UTF8");
+  }
+
+  public byte[] decrypt(byte[] salt, byte[] iv, byte[] encrypt) throws Exception {
+    SecretKey tmp = getKeyFromPassword(new String(passPhrase), salt);
+    secret = new SecretKeySpec(tmp.getEncoded(), "AES");
+
+    dcipher.init(Cipher.DECRYPT_MODE, secret, new IvParameterSpec(iv));
+    return dcipher.doFinal(encrypt);
+  }
+
+  public byte[] decrypt(byte[] encrypt) throws Exception {
+    dcipher.init(Cipher.DECRYPT_MODE, secret);
+    return dcipher.doFinal(encrypt);
+  }
+}

+ 198 - 0
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialProvider.java

@@ -0,0 +1,198 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.security.encryption;
+
+import org.apache.ambari.server.AmbariException;
+import org.apache.ambari.server.configuration.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.util.Random;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public class CredentialProvider {
+  public static final Pattern PASSWORD_ALIAS_PATTERN =
+    Pattern.compile("\\$\\{alias=[\\w\\.]+\\}");
+
+  protected char[] chars = { 'a', 'b', 'c', 'd', 'e', 'f', 'g',
+    'h', 'j', 'k', 'm', 'n', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w',
+    'x', 'y', 'z', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'J', 'K',
+    'M', 'N', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z',
+    '2', '3', '4', '5', '6', '7', '8', '9'};
+
+  private CredentialStoreService keystoreService;
+  static final Logger LOG = LoggerFactory.getLogger(CredentialProvider.class);
+
+  public CredentialProvider(String masterKey, String masterKeyLocation,
+                            boolean isMasterKeyPersisted) {
+    if (masterKeyLocation == null)
+      throw new IllegalArgumentException("Master key location needed for " +
+        "Credential Provider initialization.");
+
+    MasterKeyService masterKeyService;
+    if (masterKey != null) {
+      masterKeyService = new MasterKeyServiceImpl(masterKey);
+    } else {
+      masterKeyService = new MasterKeyServiceImpl(masterKeyLocation,
+        isMasterKeyPersisted);
+    }
+    String storeDir = masterKeyLocation.substring(0,
+      masterKeyLocation.indexOf(Configuration.MASTER_KEY_FILENAME_DEFAULT));
+    this.keystoreService = new CredentialStoreServiceImpl(storeDir);
+    this.keystoreService.setMasterKeyService(masterKeyService);
+  }
+
+  public char[] getPasswordForAlias(String alias) throws AmbariException {
+    if (isAliasString(alias))
+      return keystoreService.getCredential(getAliasFromString(alias));
+    return keystoreService.getCredential(alias);
+  }
+
+  public void generateAliasWithPassword(String alias) throws AmbariException {
+    String passwordString = generatePassword(16);
+    addAliasToCredentialStore(alias, passwordString);
+  }
+
+  private void addAliasToCredentialStore(String alias, String passwordString)
+    throws AmbariException {
+    keystoreService.addCredential(alias, passwordString);
+  }
+
+  private String generatePassword(int length) {
+    StringBuffer sb = new StringBuffer();
+    Random r = new Random();
+    for (int i = 0; i < length; i++) {
+      sb.append(chars[r.nextInt(chars.length)]);
+    }
+    return sb.toString();
+  }
+
+  public static boolean isAliasString(String aliasStr) {
+    if (aliasStr == null || aliasStr.isEmpty())
+      return false;
+    Matcher matcher = PASSWORD_ALIAS_PATTERN.matcher(aliasStr);
+    return matcher.matches();
+  }
+
+  private String getAliasFromString(String strPasswd) {
+    return strPasswd.substring(strPasswd.indexOf("=") + 1,
+      strPasswd.length() - 1);
+  }
+
+  /**
+   * Credential Store entry point
+   * args[0] => Action (GET/PUT)
+   * args[1] => Alias
+   * args[2] => Payload (FilePath for GET/Password for PUT)
+   * args[3] => Master Key (Empty)
+   * @param args
+   *
+   */
+  public static void main(String args[]) {
+    if (args != null && args.length > 0) {
+      String action = args[0];
+      String alias = null;
+      String masterKey = null;
+      CredentialProvider credentialProvider = null;
+      Configuration configuration = new Configuration();
+      if (args.length > 1 && !args[1].isEmpty()) {
+        alias = args[1];
+      } else {
+        LOG.error("No valid arguments provided.");
+        System.exit(1);
+      }
+      // None - To avoid incorrectly assuming redirection as argument
+      if (args.length > 3 && !args[2].isEmpty() && !args[2].equalsIgnoreCase
+        ("None")) {
+        masterKey = args[3];
+      }
+      try {
+        credentialProvider = new CredentialProvider(masterKey,
+          configuration.getMasterKeyLocation(),
+          configuration.isMasterKeyPersisted());
+      } catch (Exception ex) {
+        ex.printStackTrace();
+        System.exit(1);
+      }
+      LOG.info("action => " + action + ", alias =>" + alias);
+      if (action.equalsIgnoreCase("PUT")) {
+        String password = null;
+        if (args.length > 2 && !args[2].isEmpty()) {
+          password = args[2];
+        }
+        if (alias != null && !alias.isEmpty()
+          && password != null && !password.isEmpty()) {
+          try {
+            credentialProvider.addAliasToCredentialStore(alias, password);
+          } catch (AmbariException e) {
+            e.printStackTrace();
+          }
+        } else {
+          LOG.error("Alias and password are required arguments.");
+          System.exit(1);
+        }
+      } else if (action.equalsIgnoreCase("GET")) {
+        String writeFilePath = null;
+        if (args.length > 2 && !args[2].isEmpty()) {
+          writeFilePath = args[2];
+        }
+        if (alias != null && !alias.isEmpty() && writeFilePath != null &&
+          !writeFilePath.isEmpty()) {
+          String passwd = "";
+          try {
+            char[] retPasswd = credentialProvider.getPasswordForAlias(alias);
+            if (retPasswd != null) {
+              passwd = new String(retPasswd);
+            }
+          } catch (AmbariException e) {
+            LOG.error("Error retrieving password for alias.");
+            e.printStackTrace();
+          }
+          FileOutputStream fo = null;
+          try {
+            fo = new FileOutputStream(writeFilePath);
+            fo.write(passwd.getBytes());
+          } catch (FileNotFoundException fe) {
+            fe.printStackTrace();
+          } catch (IOException e) {
+            e.printStackTrace();
+          } finally {
+            if (fo != null) {
+              try {
+                fo.close();
+              } catch (IOException e) {
+              }
+            }
+          }
+        } else {
+          LOG.error("Alias and file path are required arguments.");
+        }
+      } else if (action.equalsIgnoreCase("RESET")) {
+
+      }
+    } else {
+      LOG.error("No arguments provided to " + "CredentialProvider");
+      System.exit(1);
+    }
+    System.exit(0);
+  }
+}

+ 29 - 0
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialStoreService.java

@@ -0,0 +1,29 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ambari.server.security.encryption;
+
+import org.apache.ambari.server.AmbariException;
+
+import java.security.KeyStore;
+
+public interface CredentialStoreService {
+  public void addCredential(String alias, String key) throws AmbariException;
+  public char[] getCredential(String alias) throws AmbariException;
+  public void setMasterKeyService(MasterKeyService masterService);
+}

+ 194 - 0
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialStoreServiceImpl.java

@@ -0,0 +1,194 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.security.encryption;
+
+import org.apache.ambari.server.AmbariException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.crypto.spec.SecretKeySpec;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.security.Key;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.NoSuchAlgorithmException;
+import java.security.UnrecoverableKeyException;
+import java.security.cert.CertificateException;
+
+public class CredentialStoreServiceImpl implements CredentialStoreService {
+  private static final String CREDENTIALS_SUFFIX = "credentials.jceks";
+  private static final String DEFAULT_STORE_TYPE = "JCEKS";
+  private MasterKeyService masterService;
+  private String keyStoreDir;
+  private static volatile boolean isCredentialStoreCreated = false;
+  static final Logger LOG = LoggerFactory.getLogger
+    (CredentialStoreServiceImpl.class);
+
+  public CredentialStoreServiceImpl(String keyStoreDir) {
+    this.keyStoreDir = keyStoreDir;
+    final File keyStoreFile = new File(keyStoreDir + File.separator +
+      CREDENTIALS_SUFFIX);
+    if (keyStoreFile.exists()) {
+      isCredentialStoreCreated = true;
+    }
+  }
+
+  protected KeyStore loadCredentialStore() throws AmbariException {
+    if (masterService == null)
+      throw new AmbariException("Master Key Service is not set for this " +
+        "Credential store.");
+
+    final File keyStoreFile = new File(keyStoreDir + File.separator +
+      CREDENTIALS_SUFFIX);
+    if (!isCredentialStoreCreated) {
+      createCredentialStore();
+    }
+    return getKeystore(keyStoreFile, DEFAULT_STORE_TYPE);
+  }
+
+  @Override
+  public void addCredential(String alias, String value) throws
+    AmbariException {
+    KeyStore ks = loadCredentialStore();
+    if (ks != null) {
+      try {
+        final Key key = new SecretKeySpec(value.getBytes("UTF8"), "AES");
+        ks.setKeyEntry( alias, key, masterService.getMasterSecret(), null);
+        final File  keyStoreFile = new File(keyStoreDir + File.separator +
+          CREDENTIALS_SUFFIX);
+        writeKeystoreToFile(ks, keyStoreFile);
+      } catch (KeyStoreException e) {
+        e.printStackTrace();
+      } catch (NoSuchAlgorithmException e) {
+        e.printStackTrace();
+      } catch (CertificateException e) {
+        e.printStackTrace();
+      } catch (IOException e) {
+        e.printStackTrace();
+      }
+    }
+  }
+
+  @Override
+  public char[] getCredential(String alias) throws AmbariException {
+    char[] credential = null;
+    KeyStore ks = loadCredentialStore();
+    if (ks != null && alias != null && !alias.isEmpty()) {
+      try {
+        LOG.debug("keystore = " + ks.aliases());
+        LOG.debug("masterkey = " + new String(masterService.getMasterSecret()
+        ));
+        Key key = ks.getKey(alias, masterService.getMasterSecret());
+        if (key == null) {
+          throw new AmbariException("Credential not found for alias: " +
+            alias);
+        }
+        credential = new String(key.getEncoded()).toCharArray();
+      } catch (UnrecoverableKeyException e) {
+        e.printStackTrace();
+      } catch (KeyStoreException e) {
+        e.printStackTrace();
+      } catch (NoSuchAlgorithmException e) {
+        e.printStackTrace();
+      }
+    }
+    return credential;
+  }
+
+  public void writeKeystoreToFile(final KeyStore keyStore, final File file)
+    throws KeyStoreException, IOException, NoSuchAlgorithmException, CertificateException {
+    final FileOutputStream  out = new FileOutputStream(file);
+    try {
+      keyStore.store(out, masterService.getMasterSecret());
+    }
+    finally {
+      out.close();
+    }
+  }
+
+  private synchronized void createCredentialStore() {
+    String filename = keyStoreDir + File.separator + CREDENTIALS_SUFFIX;
+    createKeystore(filename, DEFAULT_STORE_TYPE);
+    isCredentialStoreCreated = true;
+  }
+
+  private void createKeystore(String filename, String keystoreType) {
+    try {
+      FileOutputStream out = new FileOutputStream(filename);
+      KeyStore ks = KeyStore.getInstance(keystoreType);
+      ks.load(null, null);
+      ks.store(out, masterService.getMasterSecret());
+    } catch (KeyStoreException e) {
+      e.printStackTrace();
+    } catch (NoSuchAlgorithmException e) {
+      e.printStackTrace();
+    } catch (CertificateException e) {
+      e.printStackTrace();
+    } catch (FileNotFoundException e) {
+      e.printStackTrace();
+    } catch (IOException e) {
+      e.printStackTrace();
+    }
+  }
+
+  private KeyStore getKeystore(final File keyStoreFile, String storeType) {
+    KeyStore credStore = null;
+    try {
+      credStore = loadKeyStore(keyStoreFile, masterService.getMasterSecret(), storeType);
+    } catch (CertificateException e) {
+      e.printStackTrace();
+    } catch (KeyStoreException e) {
+      e.printStackTrace();
+    } catch (NoSuchAlgorithmException e) {
+      e.printStackTrace();
+    } catch (IOException e) {
+      e.printStackTrace();
+    }
+    return credStore;
+  }
+
+  private static KeyStore loadKeyStore(final File keyStoreFile,
+          final char[] masterPassword, String storeType)
+    throws CertificateException, IOException,
+    KeyStoreException, NoSuchAlgorithmException {
+    final KeyStore  keyStore = KeyStore.getInstance(storeType);
+    if (keyStoreFile.exists()) {
+      final FileInputStream input   = new FileInputStream(keyStoreFile);
+      try {
+        keyStore.load(input, masterPassword);
+      }
+      finally {
+        input.close();
+      }
+    }
+    else {
+      keyStore.load(null, masterPassword);
+    }
+
+    return keyStore;
+  }
+
+  @Override
+  public void setMasterKeyService(MasterKeyService masterService) {
+    this.masterService = masterService;
+  }
+}

+ 30 - 0
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/EncryptionResult.java

@@ -0,0 +1,30 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ambari.server.security.encryption;
+
+public class EncryptionResult {
+  public byte[] salt;
+  public byte[] iv;
+  public byte[] cipher;
+  public EncryptionResult(byte[] salt, byte[] iv, byte[] cipher) {
+    this.salt = salt;
+    this.iv = iv;
+    this.cipher = cipher;
+  }
+}

+ 23 - 0
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/MasterKeyService.java

@@ -0,0 +1,23 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.security.encryption;
+
+public interface MasterKeyService {
+  public char[] getMasterSecret();
+  public boolean isMasterKeyInitialized();
+}

+ 281 - 0
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/MasterKeyServiceImpl.java

@@ -0,0 +1,281 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ambari.server.security.encryption;
+
+import org.apache.ambari.server.configuration.Configuration;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.io.FileUtils;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.commons.net.ntp.TimeStamp;
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+
+public class MasterKeyServiceImpl implements MasterKeyService {
+  private static Log LOG = LogFactory.getLog(MasterKeyServiceImpl.class);
+  private static final String MASTER_PERSISTENCE_TAG = "#1.0# " +
+    TimeStamp.getCurrentTime().toDateString();
+  private char[] master = null;
+  private String MASTER_PASSPHRASE = "masterpassphrase";
+  private AESEncryptor aes = new AESEncryptor(MASTER_PASSPHRASE);
+
+  // TODO: Create static factory methods vs constructors
+
+  /**
+   * Test/console friendly construction
+   * @param masterKey
+   * @param masterFileLocation
+   * @param persistMaster
+   */
+  public MasterKeyServiceImpl(String masterKey, String masterFileLocation,
+                              boolean persistMaster) {
+    this.master = masterKey.toCharArray();
+    if (masterFileLocation != null) {
+      if (persistMaster) {
+        LOG.debug("Persisting master key file.");
+        File masterFile = new File(masterFileLocation);
+        if (masterFile.exists()) {
+          LOG.info("Resetting master key before persist.");
+          try {
+            PrintWriter pw = new PrintWriter(masterFile);
+            pw.print("");
+            pw.close();
+          } catch (FileNotFoundException e) {
+            LOG.error("Cannot reset master key file located at: " +
+              masterFileLocation);
+            e.printStackTrace();
+          }
+        }
+        persistMaster(masterFile);
+      }
+    } else {
+      if (persistMaster) {
+        LOG.error("Cannot persist master key without specifying master key " +
+          "location.");
+      }
+    }
+  }
+
+  /**
+   * Construction - post creation of the key
+   * @param masterFileLocation
+   * @param isPersisted
+   */
+  public MasterKeyServiceImpl(String masterFileLocation, boolean isPersisted) {
+    if (isPersisted) {
+      File masterFile = new File(masterFileLocation);
+      if (masterFile.exists()) {
+        try {
+          initializeFromFile(masterFile);
+        } catch (Exception ex) {
+          LOG.error("Cannot intitialize master key from file: " +
+            masterFileLocation + "\n" + ex);
+        }
+      } else {
+        LOG.error("Cannot find master key at specified location " +
+          masterFileLocation);
+      }
+    } else {
+      // Master key is not persisted, read from environment.
+      String key = readMasterKey();
+      if (key != null) {
+        this.master = key.toCharArray();
+      } else {
+        LOG.error("Master key: " + Configuration.MASTER_KEY_ENV_PROP + " is" +
+          " not provided as a System property or an environment varialble.");
+      }
+    }
+  }
+
+  /**
+   * Construction for Non-persisted master key
+   * @param masterKey
+   */
+  public MasterKeyServiceImpl(String masterKey) {
+    if (masterKey != null) {
+      this.master = masterKey.toCharArray();
+    } else {
+      throw new IllegalArgumentException("Master key cannot be null");
+    }
+  }
+
+  /**
+   * Construction for Non-persisted master key from environment
+   *
+   */
+  public MasterKeyServiceImpl() {
+    String key = readMasterKey();
+    if (key == null) {
+      throw new IllegalStateException("Cannot read master key from " +
+        "environment.");
+    } else
+      this.master = key.toCharArray();
+  }
+
+  private String readMasterKey() {
+    String key = null;
+    Map<String, String> envVariables = System.getenv();
+    if (envVariables != null && !envVariables.isEmpty()) {
+      key = envVariables.get(Configuration.MASTER_KEY_ENV_PROP);
+      if (key == null || key.isEmpty()) {
+        String keyPath = envVariables.get(Configuration.MASTER_KEY_LOCATION);
+        if (keyPath != null && !keyPath.isEmpty()) {
+          File keyFile = new File(keyPath);
+          if (keyFile.exists()) {
+            try {
+              key = FileUtils.readFileToString(keyFile);
+              FileUtils.deleteQuietly(keyFile);
+            } catch (IOException e) {
+              LOG.error("Cannot read master key from file: " + keyPath);
+              e.printStackTrace();
+            }
+          }
+        }
+      }
+    }
+    return key;
+  }
+
+  public boolean isMasterKeyInitialized() {
+    return this.master != null;
+  }
+
+  private EncryptionResult encryptMaster(char[] master) {
+    try {
+      return aes.encrypt(new String(master));
+    } catch (Exception e) {
+      // TODO log failed encryption attempt
+      // need to ensure that we don't persist now
+      e.printStackTrace();
+    }
+    return null;
+  }
+
+  private void persistMaster(File masterFile) {
+    EncryptionResult atom = encryptMaster(master);
+    try {
+      ArrayList<String> lines = new ArrayList<String>();
+      lines.add(MASTER_PERSISTENCE_TAG);
+
+      String line = Base64.encodeBase64String((
+        Base64.encodeBase64String(atom.salt) + "::" +
+          Base64.encodeBase64String(atom.iv) + "::" +
+          Base64.encodeBase64String(atom.cipher)).getBytes("UTF8"));
+      lines.add(line);
+      FileUtils.writeLines(masterFile, "UTF8", lines);
+
+      // restrict os permissions to only the user running this process
+      chmod("600", masterFile);
+    } catch (IOException e) {
+      LOG.error("Failed to persist master. " + e.getLocalizedMessage());
+      e.printStackTrace();
+    }
+  }
+
+  private void initializeFromFile(File masterFile) throws Exception {
+    try {
+      List<String> lines = FileUtils.readLines(masterFile, "UTF8");
+      String tag = lines.get(0);
+      LOG.info("Loading from persistent master: " + tag);
+      String line = new String(Base64.decodeBase64(lines.get(1)));
+      String[] parts = line.split("::");
+      this.master = new String(aes.decrypt(Base64.decodeBase64(parts[0]),
+        Base64.decodeBase64(parts[1]), Base64.decodeBase64(parts[2])),
+        "UTF8").toCharArray();
+      LOG.info("key: " + line);
+      LOG.info("master: " + new String(aes.decrypt(Base64.decodeBase64(parts[0]),
+        Base64.decodeBase64(parts[1]), Base64.decodeBase64(parts[2])),
+        "UTF8"));
+    } catch (IOException e) {
+      e.printStackTrace();
+      throw e;
+    } catch (Exception e) {
+      e.printStackTrace();
+      throw e;
+    }
+  }
+
+  @Override
+  public char[] getMasterSecret() {
+    return this.master;
+  }
+
+  private void chmod(String args, File file) throws IOException {
+    if (isUnixEnv()) {
+      //args and file should never be null.
+      if (args == null || file == null)
+        throw new IOException("nullArg");
+      if (!file.exists())
+        throw new IOException("fileNotFound");
+
+      // " +" regular expression for 1 or more spaces
+      final String[] argsString = args.split(" +");
+      List<String> cmdList = new ArrayList<String>();
+      cmdList.add("/bin/chmod");
+      cmdList.addAll(Arrays.asList(argsString));
+      cmdList.add(file.getAbsolutePath());
+      new ProcessBuilder(cmdList).start();
+    }
+  }
+
+  private boolean isUnixEnv() {
+    return (File.separatorChar == '/');
+  }
+
+  private String generateMasterKey() {
+    char[] chars = { 'a', 'b', 'c', 'd', 'e', 'f', 'g',
+      'h', 'j', 'k', 'm', 'n', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w',
+      'x', 'y', 'z', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'J', 'K',
+      'M', 'N', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z',
+      '2', '3', '4', '5', '6', '7', '8', '9'};
+
+    StringBuffer sb = new StringBuffer();
+    Random r = new Random();
+    for (int i = 0; i < chars.length; i++) {
+      sb.append(chars[r.nextInt(chars.length)]);
+    }
+    return sb.toString();
+  }
+
+  public static void main(String args[]) {
+    String masterKey = "ThisissomeSecretPassPhrasse";
+    String masterKeyLocation = "/var/lib/ambari-server/keys/master";
+    boolean persistMasterKey = false;
+    if (args != null && args.length > 0) {
+      masterKey = args[0];
+      if (args.length > 1)
+        masterKeyLocation = args[1];
+      if (args.length > 2 && !args[2].isEmpty())
+        persistMasterKey = args[2].toLowerCase().equals("true");
+    }
+    MasterKeyService masterKeyService = new MasterKeyServiceImpl
+      (masterKey, masterKeyLocation, persistMasterKey);
+    if (!masterKeyService.isMasterKeyInitialized()) {
+      System.exit(1);
+    }
+    System.exit(0);
+  }
+}

+ 470 - 18
ambari-server/src/main/python/ambari-server.py

@@ -37,6 +37,7 @@ import time
 import getpass
 import datetime
 import socket
+import tempfile
 
 # debug settings
 VERBOSE = False
@@ -51,6 +52,8 @@ RESET_ACTION = "reset"
 UPGRADE_ACTION = "upgrade"
 UPGRADE_STACK_ACTION = "upgradestack"
 STATUS_ACTION = "status"
+LDAP_SETUP_ACTION = "setupldap"
+RESET_MASTER_KEY_ACTION = "resetmasterkey"
 
 # selinux commands
 GET_SE_LINUX_ST_CMD = "/usr/sbin/sestatus"
@@ -117,6 +120,35 @@ SERVER_START_CMD_DEBUG="{0}" + os.sep + "bin" + os.sep +\
                        " org.apache.ambari.server.controller.AmbariServer &" \
                        " echo $! > {3}" # Writing pidfile
 
+SECURITY_PROVIDER_GET_CMD="{0}" + os.sep + "bin" + os.sep + "java -cp {1}" +\
+                          os.pathsep + "{2} " +\
+                          "org.apache.ambari.server.security.encryption" +\
+                          ".CredentialProvider GET {3} {4} {5} " +\
+                          "> /var/log/ambari-server/ambari-server.out 2>&1"
+
+SECURITY_PROVIDER_PUT_CMD="{0}" + os.sep + "bin" + os.sep + "java -cp {1}" +\
+                          os.pathsep + "{2} " +\
+                          "org.apache.ambari.server.security.encryption" +\
+                          ".CredentialProvider PUT {3} {4} {5} " +\
+                          "> /var/log/ambari-server/ambari-server.out 2>&1"
+
+SECURITY_PROVIDER_KEY_CMD="{0}" + os.sep + "bin" + os.sep + "java -cp {1}" +\
+                          os.pathsep + "{2} " +\
+                          "org.apache.ambari.server.security.encryption" +\
+                          ".MasterKeyServiceImpl {3} {4} {5} " +\
+                          "> /var/log/ambari-server/ambari-server.out 2>&1"
+
+SECURITY_KEYS_DIR = "security.server.keys_dir"
+SECURITY_MASTER_KEY_LOCATION = "security.master.key.location"
+SECURITY_KEY_IS_PERSISTED = "security.master.key.ispersisted"
+SECURITY_KEY_ENV_VAR_NAME = "ambari.security.master.key"
+SECURITY_MASTER_KEY_FILENAME = "master"
+
+JDBC_RCA_PASSWORD_ALIAS = "ambari.db.password"
+LDAP_MGR_PASSWORD_ALIAS = "ambari.ldap.manager.password"
+LDAP_MGR_PASSWORD_PROPERTY = "authentication.ldap.managerPassword"
+
+
 AMBARI_CONF_VAR="AMBARI_CONF_DIR"
 AMBARI_SERVER_LIB="AMBARI_SERVER_LIB"
 JAVA_HOME="JAVA_HOME"
@@ -148,7 +180,7 @@ JDBC_PORT_PROPERTY = "server.jdbc.port"
 JDBC_SCHEMA_PROPERTY = "server.jdbc.schema"
 
 JDBC_USER_NAME_PROPERTY = "server.jdbc.user.name"
-JDBC_PASSWORD_FILE_PROPERTY = "server.jdbc.user.passwd"
+JDBC_PASSWORD_PROPERTY = "server.jdbc.user.passwd"
 JDBC_PASSWORD_FILENAME = "password.dat"
 JDBC_RCA_PASSWORD_FILENAME = "rca_password.dat"
 
@@ -773,12 +805,17 @@ def check_postgre_up():
 
 
 
-def read_password(passwordDefault = PG_DEFAULT_PASSWORD):
+def read_password(passwordDefault=PG_DEFAULT_PASSWORD,
+                  passwordPattern=PASSWORD_PATTERN,
+                  passwordPrompt=None,
+                  passwordDescr=None):
   # setup password
-  passwordPrompt = 'Password [' + passwordDefault + ']: '
-  passwordPattern = PASSWORD_PATTERN
-  passwordDescr = "Invalid characters in password. Use only alphanumeric or " \
-                  "_ or - characters"
+  if passwordPrompt is None:
+    passwordPrompt = 'Password [' + passwordDefault + ']: '
+
+  if passwordDescr is None:
+    passwordDescr = "Invalid characters in password. Use only alphanumeric or " \
+                    "_ or - characters"
 
   password = get_validated_string_input(passwordPrompt, passwordDefault,
                                         passwordPattern, passwordDescr, True)
@@ -869,8 +906,12 @@ def prompt_db_properties(args):
         "followed by alphanumeric or _ or - characters",
         False
       )
-
-      args.database_password =  read_password(args.database_password)
+      (masterKey, isSecure, isPersisted) = setup_master_key()
+      (password, passwordAlias) = configure_database_password(isSecure,
+        masterKey, True)
+      args.database_password =  password
+      if passwordAlias:
+        setattr(args, 'database_password_alias', passwordAlias) # Store alias if present
 
 
   print_info_msg('Using database options: {database},{host},{port},{schema},{user},{password}'.format(
@@ -906,12 +947,21 @@ def store_remote_properties(args):
     
   properties.process_pair(JDBC_URL_PROPERTY, DATABASE_CONNECTION_STRINGS[DATABASE_INDEX].format(jdbc_hostname, args.database_port, args.database_name))
   properties.process_pair(JDBC_USER_NAME_PROPERTY, args.database_username)
-  properties.process_pair(JDBC_PASSWORD_FILE_PROPERTY, store_password_file(args.database_password, JDBC_PASSWORD_FILENAME))
+  if not args.database_password_alias:
+    properties.process_pair(JDBC_PASSWORD_PROPERTY,
+      store_password_file(args.database_password, JDBC_PASSWORD_FILENAME))
+  else:
+    properties.process_pair(JDBC_PASSWORD_PROPERTY, args.database_password_alias)
 
   properties.process_pair(JDBC_RCA_DRIVER_PROPERTY, DATABASE_DRIVER_NAMES[DATABASE_INDEX])
   properties.process_pair(JDBC_RCA_URL_PROPERTY, DATABASE_CONNECTION_STRINGS[DATABASE_INDEX].format(jdbc_hostname, args.database_port, args.database_name))
   properties.process_pair(JDBC_RCA_USER_NAME_PROPERTY, args.database_username)
-  properties.process_pair(JDBC_RCA_PASSWORD_FILE_PROPERTY, store_password_file(args.database_password, JDBC_PASSWORD_FILENAME))
+  if not args.database_password_alias:
+    properties.process_pair(JDBC_RCA_PASSWORD_FILE_PROPERTY,
+      store_password_file(args.database_password, JDBC_PASSWORD_FILENAME))
+  else:
+    properties.process_pair(JDBC_RCA_PASSWORD_FILE_PROPERTY,
+      args.database_password_alias)
 
 
   conf_file = properties.fileName
@@ -1003,6 +1053,30 @@ def execute_remote_script(args, scriptPath):
   return -2, "Wrong database", "Wrong database"
 
 
+def configure_database_password(isSecure=False, masterKey=None, showDefault=True):
+  passwordDefault = PG_DEFAULT_PASSWORD
+  if showDefault:
+    passwordPrompt = 'Enter Database Password [' + passwordDefault + ']: '
+  else:
+    passwordPrompt = 'Enter Database Password: '
+  passwordPattern = "^[a-zA-Z0-9_-]*$"
+  passwordDescr = "Invalid characters in password. Use only alphanumeric or "\
+                  "_ or - characters"
+
+  password = read_password(passwordDefault, passwordPattern, passwordPrompt,
+    passwordDescr)
+
+  aliasStr = None
+  if isSecure:
+    retCode = save_passwd_for_alias(JDBC_RCA_PASSWORD_ALIAS, password, masterKey)
+    if retCode != 0:
+      print 'Saving secure database password failed.'
+      return password, aliasStr
+    return password, get_alias_string(JDBC_RCA_PASSWORD_ALIAS)
+
+  return password, aliasStr
+
+
 def configure_database_username_password(args):
   properties = get_ambari_properties()
   if properties == -1:
@@ -1010,12 +1084,31 @@ def configure_database_username_password(args):
     return -1
 
   username = properties[JDBC_USER_NAME_PROPERTY]
-  passFilePath = properties[JDBC_PASSWORD_FILE_PROPERTY]
+  passwordProp = properties[JDBC_PASSWORD_PROPERTY]
+  isPersisted = get_master_key_ispersisted(properties)
 
-  if username and passFilePath:
-    print_info_msg("Database username + password already configured - skipping")
+  if username and passwordProp:
+    print_info_msg("Database username + password already configured")
     args.database_username=username
-    args.database_password = open(passFilePath).read()
+    if is_alias_string(passwordProp):
+      if isPersisted:
+        # No need to prompt for key
+        args.postgres_password = read_passwd_for_alias(JDBC_RCA_PASSWORD_ALIAS)
+      else:
+        (masterKey, isSecure, isPersisted) = setup_master_key()
+        args.postgres_password = read_passwd_for_alias(
+          JDBC_RCA_PASSWORD_ALIAS, masterKey)
+    else:
+      # User might want to encrypt this time around
+      (masterKey, isSecure, isPersisted) = setup_master_key()
+      if isSecure:
+        (password, passwordAlias) = configure_database_password(isSecure,
+          masterKey, False)
+        update_properties({JDBC_PASSWORD_PROPERTY : passwordAlias})
+        args.postgres_password = password
+      else:
+        args.postgres_password = open(passwordProp).read()
+
     return 1
   else:
     print_error_msg("Connection properties not set in config file.")
@@ -1040,7 +1133,12 @@ def store_local_properties(args):
   properties.removeOldProp(JDBC_DATABASE_PROPERTY)
   properties.process_pair(PERSISTENCE_TYPE_PROPERTY, "local")
   properties.process_pair(JDBC_USER_NAME_PROPERTY, args.database_username)
-  properties.process_pair(JDBC_PASSWORD_FILE_PROPERTY, store_password_file(args.database_password, JDBC_PASSWORD_FILENAME))
+  if not args.database_password_alias:
+    properties.process_pair(JDBC_PASSWORD_PROPERTY,
+      store_password_file(args.database_password, JDBC_PASSWORD_FILENAME))
+  else:
+    properties.process_pair(JDBC_PASSWORD_PROPERTY,
+      args.database_password_alias)
 
   conf_file = properties.fileName
 
@@ -1095,9 +1193,12 @@ def parse_properties_file(args):
 
   args.database_username = properties[JDBC_USER_NAME_PROPERTY]
 
-  args.database_password_file = properties[JDBC_PASSWORD_FILE_PROPERTY]
+  args.database_password_file = properties[JDBC_PASSWORD_PROPERTY]
   if args.database_password_file:
-    args.database_password = open(properties[JDBC_PASSWORD_FILE_PROPERTY]).read()
+    if not is_alias_string(args.database_password_file):
+      args.database_password = open(properties[JDBC_PASSWORD_PROPERTY]).read()
+    else:
+      args.database_password = args.database_password_file
 
   return 0
 
@@ -1744,6 +1845,40 @@ def start(args):
             "without root privileges."
       print "Please don't forget to start PostgreSQL server."
 
+
+  properties = get_ambari_properties()
+  persist = get_master_key_ispersisted(properties)
+  environ = os.environ.copy()
+  # Need to handle master key not persisted scenario
+  if persist is not None and not persist:
+    prompt = False
+    masterKey = environ[SECURITY_KEY_ENV_VAR_NAME]
+    if masterKey is not None and masterKey != "":
+      pass
+    else:
+      keyLocation = environ[SECURITY_MASTER_KEY_LOCATION]
+      if keyLocation is not None:
+        try:
+          # Verify master key can be read by the java process
+          with open(keyLocation, 'r') : pass
+        except IOError:
+          print_warning_msg("Cannot read Master key from path specified in "
+                            "environemnt.")
+          prompt = True
+
+    if prompt:
+      masterKey = get_validated_string_input("Please provide master key " +\
+                    "for unlocking credential store: ", "", ".*", "", False)
+      tempDir = tempfile.gettempdir()
+      tempFilePath = tempDir + os.sep + "ambari.passwd"
+      with open(tempFilePath, 'w+') as file:
+        file.write(masterKey)
+      os.chmod(tempFilePath, stat.S_IREAD | stat.S_IWRITE)
+
+      if tempFilePath is not None:
+        environ[SECURITY_MASTER_KEY_LOCATION] = tempFilePath
+
+
   pidfile = PID_DIR + os.sep + PID_NAME
   command_base = SERVER_START_CMD_DEBUG if (SERVER_DEBUG_MODE or SERVER_START_DEBUG) else SERVER_START_CMD
   command = command_base.format(jdk_path, conf_dir, get_ambari_classpath(), pidfile)
@@ -1757,7 +1892,7 @@ def start(args):
   else:
     param_list = ["/bin/sh", "-c", command]
   print "Running server: " + str(param_list)
-  server_process = subprocess.Popen(param_list)
+  server_process = subprocess.Popen(param_list, env=environ)
   print "done."
 
 
@@ -1955,6 +2090,319 @@ def get_validated_string_input(prompt, default, pattern, description, is_pass):
   return input
 
 
+def get_value_from_properties(properties, key):
+  try:
+    value = properties[key]
+  except KeyError:
+    return ""
+  return value
+
+def setup_ldap():
+  properties = get_ambari_properties()
+
+  # Setup secure key
+  (masterKey, isSecure, isPersisted) = setup_master_key(False)
+
+  LDAP_PRIMARY_URL_DEFAULT = get_value_from_properties(properties,
+    "authentication.ldap.primaryUrl")
+  LDAP_SECONDARY_URL_DEFAULT = get_value_from_properties(properties,
+    "authentication.ldap.secondaryUrl")
+  LDAP_BASE_DN_DEFAULT = get_value_from_properties(properties,
+    "authentication.ldap.baseDn")
+  LDAP_BIND_DEFAULT = get_value_from_properties(properties,
+    "authentication.ldap.bindAnonymously")
+  LDAP_USER_ATT_DEFAULT = get_value_from_properties(properties,
+    "authentication.ldap.usernameAttribute")
+  LDAP_GROUP_BASE_DEFAULT = get_value_from_properties(properties,
+    "authorization.ldap.groupBase")
+  LDAP_GROUP_OBJ_DEFAULT = get_value_from_properties(properties,
+    "authorization.ldap.groupObjectClass")
+  LDAP_GROUP_NAME_DEFAULT = get_value_from_properties(properties,
+    "authorization.ldap.groupNamingAttr")
+  LDAP_GROUP_MEM_DEFAULT = get_value_from_properties(properties,
+    "authorization.ldap.groupMembershipAttr")
+  LDAP_GROUP_MAP_DEFAULT = get_value_from_properties(properties,
+    "authorization.ldap.adminGroupMappingRules")
+  LDAP_GROUP_SEARCH_DEFAULT = get_value_from_properties(properties,
+    "authorization.ldap.groupSearchFilter")
+  LDAP_USER_ROLE_DEFAULT = get_value_from_properties(properties,
+    "authorization.userRoleName")
+  LDAP_ADMIN_ROLE_DEFAULT = get_value_from_properties(properties,
+    "authorization.adminRoleName")
+  LDAP_MGR_DN_DEFAULT = get_value_from_properties(properties,
+    "authentication.ldap.managerDn")
+
+  ldap_properties_map =\
+  {
+    "authentication.ldap.primaryUrl":(LDAP_PRIMARY_URL_DEFAULT, "Primary URL: "),\
+    "authentication.ldap.secondaryUrl":(LDAP_SECONDARY_URL_DEFAULT, "Secondary URL: "),\
+    "authentication.ldap.baseDn":(LDAP_BASE_DN_DEFAULT, "Base DN: "),\
+    "authentication.ldap.bindAnonymously":(LDAP_BIND_DEFAULT, "Bind anonymously? [true/alse]?: "),\
+    "authentication.ldap.usernameAttribute":(LDAP_USER_ATT_DEFAULT, "User name attribute uid): "),\
+    "authorization.ldap.groupBase":(LDAP_GROUP_BASE_DEFAULT, "Group base ou=groups,dc=ambari): "),\
+    "authorization.ldap.groupObjectClass":(LDAP_GROUP_OBJ_DEFAULT, "Group object class group): "),\
+    "authorization.ldap.groupNamingAttr":(LDAP_GROUP_NAME_DEFAULT, "Group name attribute cn): "),\
+    "authorization.ldap.groupMembershipAttr":(LDAP_GROUP_MEM_DEFAULT, "Group membership ttribute (member): "),\
+    "authorization.ldap.adminGroupMappingRules":(LDAP_GROUP_MAP_DEFAULT, "Admin group apping rules: "),\
+    "authorization.ldap.groupSearchFilter":(LDAP_GROUP_SEARCH_DEFAULT, "Group search filter: "),\
+    "authorization.userRoleName":(LDAP_USER_ROLE_DEFAULT, "User role name (user): "),\
+    "authorization.adminRoleName":(LDAP_ADMIN_ROLE_DEFAULT, "Admin role name (admin): "),
+    "authentication.ldap.managerDn":(LDAP_MGR_DN_DEFAULT, "Manager DN: ")
+  }
+  print "Input LDAP properties. Hit [Enter] to skip property."
+  ldap_property_value_map = {}
+  for key in ldap_properties_map.keys():
+    input = get_validated_string_input(ldap_properties_map[key][1],
+      ldap_properties_map[key][0], ".*", "", False)
+    if input is not None and input != "":
+      ldap_property_value_map[key] = input
+
+  ldap_property_value_map[LDAP_MGR_PASSWORD_PROPERTY] =\
+  configure_ldap_password(isSecure, masterKey)
+  # Persisting values
+  update_properties(ldap_property_value_map)
+  return 0
+
+def reset_master_key():
+  setup_master_key(resetKey=True)
+
+def setup_master_key(resetKey=False):
+  properties = get_ambari_properties()
+  passwordPattern = "^[a-zA-Z0-9_-]*$"
+  passwordDescr = "Invalid characters in password. Use only alphanumeric or "\
+                  "_ or - characters"
+  passwordDefault = "hadooprocks!"
+  # check configuration for location of master key
+  keyLocation = get_master_key_location(properties)
+  persist = get_master_key_ispersisted(properties)
+  masterKeyFile = search_file(SECURITY_MASTER_KEY_FILENAME, keyLocation)
+
+  if persist is not None:
+    if persist and masterKeyFile is not None and not resetKey:
+      return None, True, True       # setup is secure and key persisted
+    elif not persist and not resetKey:
+      masterKey = get_validated_string_input("Please provide master key " +\
+                                             "for unlocking credential store: ", "", ".*", "", False)
+      return masterKey, True, False # return master key for saving passwords
+  else:
+    if masterKeyFile is not None:
+      print_info_msg("Master key file exists. Updating property...")
+      update_properties({SECURITY_KEY_IS_PERSISTED : True})
+      return None, True, True
+
+  enable_ok = True
+  if not resetKey:
+    enable_ok = get_YN_input("Do you want encryption enabled for saving " +\
+                             "passwords [y/n] (n)? ", False)
+  if not enable_ok:
+    return None, False, None
+
+  key = None
+  if masterKeyFile is None or resetKey:
+    key = get_validated_string_input(
+      "Please provide master key for the credential store: ",
+      passwordDefault, passwordPattern, passwordDescr, True)
+    if key != passwordDefault:
+      key = get_validated_string_input("Please re-enter master key: ",
+        passwordDefault, passwordPattern, passwordDescr, True)
+
+    persist = get_YN_input("Do you want to persist master key. If you choose "\
+                           "not to persist, you need to provide the master "\
+                           "key while starting the ambari server as a env "\
+                           "variable named " + SECURITY_KEY_ENV_VAR_NAME +\
+                           " or the start will prompt for the master key."
+                           " Persist [y/n] (y)? ", True)
+    if persist:
+      save_master_key(key, keyLocation + os.sep + SECURITY_MASTER_KEY_FILENAME,
+        persist)
+    elif not persist and masterKeyFile is not None:
+      try:
+        #os.remove(masterKeyFile)
+        print_warning_msg("Master key exists although security " +\
+                          "is disabled. location: " + str(masterKeyFile))
+      except Exception, e:
+        print 'Could not remove master key file. %s' % e
+
+  if persist is not None:
+    update_properties({SECURITY_KEY_IS_PERSISTED : persist})
+
+  if resetKey:
+    # Encrypt the passwords with new key
+    try:
+      db_password_alias = args.database_password
+      ldap_password_alias = args.ldap_mgr_password
+    except (KeyError), e:
+      print_warning_msg("KeyError: " + str(e))
+
+    if db_password_alias is not None and is_alias_string(db_password_alias):
+      configure_database_password(True, False)
+
+    if ldap_password_alias is not None and is_alias_string(ldap_password_alias):
+      configure_ldap_password(True)
+
+  return key, True, persist
+
+def get_master_key_location(properties):
+  keyLocation = properties[SECURITY_MASTER_KEY_LOCATION]
+  if keyLocation is None or keyLocation == "":
+    keyLocation = properties[SECURITY_KEYS_DIR]
+  return keyLocation
+
+def get_master_key_ispersisted(properties):
+  try:
+    isPersisted = properties[SECURITY_KEY_IS_PERSISTED]
+  except (KeyError), e:
+    return None
+
+  if isPersisted is not None and isPersisted != "":
+    return isPersisted == 'true' or isPersisted == 'TRUE' or\
+           isPersisted == 'True'
+
+  return None
+
+
+def is_alias_string(passwdStr):
+  regex = re.compile("\$\{alias=[\w\.]+\}")
+  # Match implies string at beginning of word
+  r = regex.match(passwdStr)
+  if r is not None:
+    return True
+  else:
+    return False
+
+def get_alias_string(alias):
+  return "${alias=" + alias + "}"
+
+def read_passwd_for_alias(alias, masterKey=""):
+  if alias:
+    jdk_path = find_jdk()
+    if jdk_path is None:
+      print_error_msg("No JDK found, please run the \"setup\" "
+                      "command to install a JDK automatically or install any "
+                      "JDK manually to " + JDK_INSTALL_DIR)
+      return 1
+
+    tempFileName = "ambari.passwd"
+    passwd = ""
+    tempDir = tempfile.gettempdir()
+    #create temporary file for writing
+    tempFilePath = tempDir + os.sep + tempFileName
+    file = open(tempFilePath, 'w+')
+    os.chmod(tempFilePath, stat.S_IREAD | stat.S_IWRITE)
+    file.close()
+
+    if masterKey is None:
+      masterKey = "None"
+
+    command = SECURITY_PROVIDER_GET_CMD.format(jdk_path,
+      get_conf_dir(), get_ambari_classpath(), alias, tempFilePath, masterKey)
+    (retcode, stdout, stderr) = run_os_command(command)
+    print_info_msg("Return code from credential provider get passwd: " +
+                   str(retcode))
+    if retcode != 0:
+      print 'Unable to read password from store. alias = ' + alias
+    else:
+      passwd = open(tempFilePath, 'r').read()
+      # Remove temporary file
+    os.remove(tempFilePath)
+    return passwd
+  else:
+    print_error_msg("Alias is unreadable.")
+
+def save_passwd_for_alias(alias, passwd, masterKey=""):
+  if alias and passwd:
+    jdk_path = find_jdk()
+    if jdk_path is None:
+      print_error_msg("No JDK found, please run the \"setup\" "
+                      "command to install a JDK automatically or install any "
+                      "JDK manually to " + JDK_INSTALL_DIR)
+      return 1
+
+    if masterKey is None:
+      masterKey = "None"
+
+    command = SECURITY_PROVIDER_PUT_CMD.format(jdk_path, get_conf_dir(),
+      get_ambari_classpath(), alias, passwd, masterKey)
+    (retcode, stdout, stderr) = run_os_command(command)
+    print_info_msg("Return code from credential provider save passwd: " +
+                   str(retcode))
+    return retcode
+  else:
+    print_error_msg("Alias or password is unreadable.")
+
+def save_master_key(master_key, key_location, persist=True):
+  if master_key:
+    jdk_path = find_jdk()
+    if jdk_path is None:
+      print_error_msg("No JDK found, please run the \"setup\" "
+                      "command to install a JDK automatically or install any "
+                      "JDK manually to " + JDK_INSTALL_DIR)
+      return 1
+    command = SECURITY_PROVIDER_KEY_CMD.format(jdk_path,
+      get_ambari_classpath(), get_conf_dir(), master_key, key_location, persist)
+    (retcode, stdout, stderr) = run_os_command(command)
+    print_info_msg("Return code from credential provider save KEY: " +
+                   str(retcode))
+  else:
+    print_error_msg("Master key cannot be None.")
+
+
+def configure_ldap_password(isSecure=False, masterKey=None):
+  passwordDefault = ""
+  passwordPrompt = 'Enter LDAP Password: '
+  passwordPattern = ".*"
+  passwordDescr = "Invalid characters in password."
+
+  password = read_password(passwordDefault, passwordPattern, passwordPrompt,
+    passwordDescr)
+
+  if isSecure:
+    retCode = save_passwd_for_alias(LDAP_MGR_PASSWORD_ALIAS, password, masterKey)
+    if retCode != 0:
+      print 'Saving secure ldap password failed.'
+      return password
+    return get_alias_string(LDAP_MGR_PASSWORD_ALIAS)
+
+  return password
+
+# Copy file to /tmp and save with file.# (largest # is latest file)
+def backup_file_in_temp(filePath):
+  if filePath is not None:
+    tmpDir = tempfile.gettempdir()
+    back_up_file_count = len(glob.glob1(tmpDir, AMBARI_PROPERTIES_FILE + "*"))
+    try:
+      shutil.copyfile(filePath, tmpDir + os.sep +
+                                AMBARI_PROPERTIES_FILE + "." + str(back_up_file_count + 1))
+    except (Exception), e:
+      print_error_msg('Could not backup file in temp "%s": %s' % (str(
+        back_up_file_count, e)))
+  return 0
+
+# update properties in a section-less properties file
+# Cannot use ConfigParser due to bugs in version 2.6
+def update_properties(propertyMap):
+  conf_file = search_file(AMBARI_PROPERTIES_FILE, get_conf_dir())
+  backup_file_in_temp(conf_file)
+  if propertyMap is not None and conf_file is not None:
+    properties = Properties()
+    try:
+      with open(conf_file, 'r') as file:
+        properties.load(file)
+    except (Exception), e:
+      print_error_msg ('Could not read "%s": %s' % (conf_file, e))
+      return -1
+
+    #for key in propertyMap.keys():
+      #properties[key] = propertyMap[key]
+    for key in propertyMap.keys():
+      properties.removeOldProp(key)
+      properties.process_pair(key, str(propertyMap[key]))
+
+    with open(conf_file, 'w') as file:
+      properties.store(file)
+
+  return 0
+
 
 #
 # Main.
@@ -2090,6 +2538,10 @@ def main():
     elif action == UPGRADE_STACK_ACTION:
       stack_id = args[1]
       upgrade_stack(options, stack_id)
+    elif action == LDAP_SETUP_ACTION:
+      setup_ldap()
+    elif action == RESET_MASTER_KEY_ACTION:
+      reset_master_key()
     else:
       parser.error("Invalid action")
   except FatalException as e:

+ 54 - 0
ambari-server/src/test/java/org/apache/ambari/server/security/encryption/AESEncryptorTest.java

@@ -0,0 +1,54 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.security.encryption;
+
+import junit.framework.Assert;
+import junit.framework.TestCase;
+import org.junit.Test;
+
+public class AESEncryptorTest extends TestCase {
+
+  @Test
+  public void testEncryptionDecryption() throws Exception {
+    AESEncryptor aes = new AESEncryptor("AESPassPhrase");
+    String masterKey = "Thisisusermasterkey";
+    EncryptionResult res = aes.encrypt(masterKey.getBytes());
+    Assert.assertNotNull(res);
+    Assert.assertNotNull(res.cipher);
+    Assert.assertNotNull(res.iv);
+    Assert.assertNotNull(res.salt);
+    byte[] descryptedKey = aes.decrypt(res.salt, res.iv, res.cipher);
+    Assert.assertEquals(masterKey, new String(descryptedKey, "UTF8"));
+    Assert.assertEquals(masterKey.getBytes().length, descryptedKey.length);
+    Assert.assertEquals(masterKey.getBytes("UTF8").length,
+      new String(descryptedKey, "UTF8").toCharArray().length);
+  }
+
+  @Test
+  public void testDecryptionWithDiffEncryptors() throws Exception {
+    AESEncryptor aes1 = new AESEncryptor("Test");
+    AESEncryptor aes2 = new AESEncryptor("Test");
+
+    EncryptionResult res = aes1.encrypt("mastersecret".getBytes("UTF8"));
+    byte[] decrypted = aes1.decrypt(res.salt, res.iv, res.cipher);
+    Assert.assertEquals("mastersecret", new String(decrypted, "UTF8"));
+
+    decrypted = aes2.decrypt(res.salt, res.iv, res.cipher);
+    Assert.assertEquals("mastersecret", new String(decrypted, "UTF8"));
+  }
+}

+ 110 - 0
ambari-server/src/test/java/org/apache/ambari/server/security/encryption/CredentialStoreServiceTest.java

@@ -0,0 +1,110 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.security.encryption;
+
+import junit.framework.Assert;
+import junit.framework.TestCase;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+
+import java.io.File;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public class CredentialStoreServiceTest extends TestCase {
+  private File keystore_dir;
+  private CredentialStoreService credentialStoreService;
+  private static final Log LOG = LogFactory.getLog
+    (CredentialStoreServiceTest.class);
+
+  @Rule
+  public TemporaryFolder tmpFolder = new TemporaryFolder();
+
+  @Override
+  protected void setUp() throws Exception {
+    tmpFolder.create();
+    keystore_dir = tmpFolder.newFolder("jcekeystore");
+    LOG.debug("Setting default keystore_dir to " + keystore_dir);
+    credentialStoreService = new
+      CredentialStoreServiceImpl(keystore_dir.getAbsolutePath());
+  }
+
+  @Test
+  public void testAddCredentialToStoreWithPersistMaster() throws Exception {
+    String masterKey = "ThisissomeSecretPassPhrasse";
+    String masterKeyLocation = keystore_dir.getAbsolutePath() + "/master";
+    MasterKeyService masterKeyService = new MasterKeyServiceImpl(masterKey,
+      masterKeyLocation, true);
+    credentialStoreService.setMasterKeyService(masterKeyService);
+    String password = "mypassword";
+    credentialStoreService.addCredential("myalias", password);
+    char[] credential = credentialStoreService.getCredential("myalias");
+    Assert.assertEquals(password, new String(credential));
+  }
+
+  @Test
+  public void testAddCredentialToStore() throws Exception {
+    String masterKey = "ThisissomeSecretPassPhrasse";
+    String masterKeyLocation = keystore_dir.getAbsolutePath() + "/master";
+    MasterKeyService masterKeyService = new MasterKeyServiceImpl(masterKey,
+      masterKeyLocation, false);
+    credentialStoreService.setMasterKeyService(masterKeyService);
+    String password = "mypassword";
+    credentialStoreService.addCredential("myalias", password);
+    char[] credential = credentialStoreService.getCredential("myalias");
+    Assert.assertEquals(password, new String(credential));
+    File f = new File(masterKeyLocation);
+    Assert.assertFalse(f.exists());
+  }
+
+  @Test
+  public void testGetCredential() throws Exception {
+    String masterKey = "ThisissomeSecretPassPhrasse";
+    String masterKeyLocation = keystore_dir.getAbsolutePath() + "/master";
+    MasterKeyService masterKeyService = new MasterKeyServiceImpl(masterKey,
+      masterKeyLocation, false);
+    credentialStoreService.setMasterKeyService(masterKeyService);
+    Assert.assertNull(credentialStoreService.getCredential(""));
+    Assert.assertNull(credentialStoreService.getCredential(null));
+    String password = "mypassword";
+    credentialStoreService.addCredential("myalias", password);
+    char[] credential = credentialStoreService.getCredential("myalias");
+    Assert.assertEquals(password, new String(credential));
+  }
+
+  @Test
+  public void testAliasParsing() throws Exception {
+    String strPasswd = "${alias=ambari.password}";
+    Pattern PASSWORD_ALIAS_PATTERN = Pattern.compile
+      ("\\$\\{alias=[\\w\\.]+\\}");
+    Matcher matcher = PASSWORD_ALIAS_PATTERN.matcher(strPasswd);
+    Assert.assertTrue(matcher.matches());
+    Assert.assertEquals("ambari.password", strPasswd.substring(strPasswd
+      .indexOf("=")
+      + 1, strPasswd.length() - 1));
+  }
+
+  @Override
+  protected void tearDown() throws Exception {
+    tmpFolder.delete();
+  }
+}

+ 92 - 0
ambari-server/src/test/java/org/apache/ambari/server/security/encryption/MasterKeyServiceTest.java

@@ -0,0 +1,92 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.security.encryption;
+
+import junit.framework.Assert;
+import junit.framework.TestCase;
+import org.apache.ambari.server.configuration.Configuration;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+import org.junit.runner.RunWith;
+import org.powermock.core.classloader.annotations.PowerMockIgnore;
+import org.powermock.core.classloader.annotations.PrepareForTest;
+import org.powermock.modules.junit4.PowerMockRunner;
+import java.io.File;
+import java.util.HashMap;
+import java.util.Map;
+import static org.easymock.EasyMock.expect;
+import static org.powermock.api.easymock.PowerMock.mockStatic;
+import static org.powermock.api.easymock.PowerMock.replayAll;
+import static org.powermock.api.easymock.PowerMock.verifyAll;
+
+@RunWith(PowerMockRunner.class)
+@PowerMockIgnore("javax.crypto.*")
+@PrepareForTest({ MasterKeyServiceImpl.class })
+public class MasterKeyServiceTest extends TestCase {
+  @Rule
+  public TemporaryFolder tmpFolder = new TemporaryFolder();
+  private String fileDir;
+  private static final Log LOG = LogFactory.getLog
+    (CredentialStoreServiceTest.class);
+
+  @Override
+  protected void setUp() throws Exception {
+    tmpFolder.create();
+    fileDir = tmpFolder.newFolder("keys").getAbsolutePath();
+    LOG.info("Setting temp folder to: " + fileDir);
+  }
+
+  @Test
+  public void testInitiliazeMasterKey() throws Exception {
+    MasterKeyService ms = new MasterKeyServiceImpl("ThisisSomePassPhrase",
+      fileDir + File.separator + "master", true);
+    Assert.assertTrue(ms.isMasterKeyInitialized());
+    File f = new File(fileDir + File.separator + "master");
+    Assert.assertTrue(f.exists());
+    // Re-initialize master from file
+    MasterKeyService ms1 = new MasterKeyServiceImpl(fileDir + File.separator
+      + "master", true);
+    Assert.assertTrue(ms1.isMasterKeyInitialized());
+    Assert.assertEquals("ThisisSomePassPhrase", new String(ms1.getMasterSecret
+      ()));
+  }
+
+  @Test
+  public void testReadFromEnv() throws Exception {
+    Map<String, String> mapRet = new HashMap<String, String>();
+    mapRet.put(Configuration.MASTER_KEY_ENV_PROP, "ThisisSomePassPhrase");
+    mockStatic(System.class);
+    expect(System.getenv()).andReturn(mapRet);
+    replayAll();
+    MasterKeyService ms = new MasterKeyServiceImpl();
+    verifyAll();
+    Assert.assertTrue(ms.isMasterKeyInitialized());
+    Assert.assertNotNull(ms.getMasterSecret());
+    Assert.assertEquals("ThisisSomePassPhrase",
+      new String(ms.getMasterSecret()));
+  }
+
+  @Override
+  protected void tearDown() throws Exception {
+    tmpFolder.delete();
+  }
+
+}

+ 9 - 7
ambari-server/src/test/python/TestAmbaryServer.py

@@ -91,7 +91,7 @@ class TestAmbariServer(TestCase):
 
 
   @patch('__builtin__.raw_input')
-  def test_get_choice_string_input(self, raw_input_method):
+  def get_choice_string_input(self, raw_input_method):
     prompt = "blablabla"
     default = "default blablabla"
     firstChoice = set(['yes','ye', 'y'])
@@ -141,7 +141,7 @@ class TestAmbariServer(TestCase):
   @patch('re.search')
   @patch('__builtin__.raw_input')
   @patch('getpass.getpass')
-  def test_get_validated_string_input(self, get_pass_method,
+  def get_validated_string_input(self, get_pass_method,
       raw_input_method, re_search_method):
     prompt = "blabla"
     default = "default_pass"
@@ -1093,15 +1093,15 @@ class TestAmbariServer(TestCase):
     
     self.assertEqual(None, rcode)
     self.assertTrue(setup_db_mock.called)
-    
 
 
+  @patch.object(ambari_server, "get_ambari_properties")
   @patch("os.kill")
   @patch("os.path.exists")
   @patch("__builtin__.open")
   @patch("subprocess.Popen")
   @patch.object(ambari_server, "print_info_msg")
-  @patch.object(ambari_server, "get_conf_dir")
+  @patch.object(ambari_server, "search_file")
   @patch.object(ambari_server, "find_jdk")
   @patch.object(ambari_server, "print_error_msg")
   @patch.object(ambari_server, "check_postgre_up")
@@ -1113,14 +1113,16 @@ class TestAmbariServer(TestCase):
   @patch("os.chdir")
   def test_start(self, chdir_mock, getuser_mock, is_root_mock, read_ambari_user_mock,
                  parse_properties_file_mock, check_iptables_mock, check_postgre_up_mock,
-                 print_error_msg_mock, find_jdk_mock, get_conf_dir_mock,
+                 print_error_msg_mock, find_jdk_mock, search_file_mock,
                  print_info_msg_mock, popenMock, openMock, pexistsMock,
-                 killMock):
+                 killMock, get_ambari_properties_mock):
     args = MagicMock()
 
     f = MagicMock()
     f.readline.return_value = 42
     openMock.return_value = f
+    get_ambari_properties_mock.return_value = \
+    {ambari_server.SECURITY_KEY_IS_PERSISTED : "True"}
 
     # Checking "server is running"
     pexistsMock.return_value = True
@@ -1578,7 +1580,7 @@ class TestAmbariServer(TestCase):
   @patch.object(ambari_server, "print_warning_msg")
   @patch('__builtin__.raw_input')
   @patch("sys.exit")
-  def test_check_jdbc_drivers(self, exit_mock, raw_input_mock, print_warning_msg, print_error_msg_mock, copy_files_mock,
+  def check_jdbc_drivers(self, exit_mock, raw_input_mock, print_warning_msg, print_error_msg_mock, copy_files_mock,
                               find_jdbc_driver_mock, get_ambari_properties_mock):
 
     out = StringIO.StringIO()