|
|
@@ -16,47 +16,72 @@ Apache Hadoop ${project.version}
|
|
|
================================
|
|
|
|
|
|
Apache Hadoop ${project.version} is a minor release in the 2.x.y release
|
|
|
-line, building upon the previous stable release 2.4.1.
|
|
|
+line, building upon the previous stable release 2.7.3.
|
|
|
|
|
|
Here is a short overview of the major features and improvements.
|
|
|
|
|
|
* Common
|
|
|
|
|
|
- * Authentication improvements when using an HTTP proxy server. This is
|
|
|
- useful when accessing WebHDFS via a proxy server.
|
|
|
+ * Support async call retry and failover which can be used in async DFS
|
|
|
+ implementation with retry effort.
|
|
|
|
|
|
- * A new Hadoop metrics sink that allows writing directly to Graphite.
|
|
|
+ * Cross Frame Scripting (XFS) prevention for UIs can be provided through
|
|
|
+ a common servlet filter.
|
|
|
|
|
|
- * [Specification work](./hadoop-project-dist/hadoop-common/filesystem/index.html)
|
|
|
- related to the Hadoop Compatible Filesystem (HCFS) effort.
|
|
|
+ * S3A improvements: add ability to plug in any AWSCredentialsProvider,
|
|
|
+ support read s3a credentials from hadoop credential provider API in
|
|
|
+ addition to XML configuraiton files, support Amazon STS temporary
|
|
|
+ credentials
|
|
|
+
|
|
|
+ * WASB improvements: adding append API support
|
|
|
+
|
|
|
+ * Build enhancements: replace dev-support with wrappers to Yetus,
|
|
|
+ provide a docker based solution to setup a build environment,
|
|
|
+ remove CHANGES.txt and rework the change log and release notes.
|
|
|
+
|
|
|
+ * Add posixGroups support for LDAP groups mapping service.
|
|
|
|
|
|
* HDFS
|
|
|
|
|
|
- * Support for POSIX-style filesystem extended attributes. See the
|
|
|
- [user documentation](./hadoop-project-dist/hadoop-hdfs/ExtendedAttributes.html)
|
|
|
- for more details.
|
|
|
+ * WebHDFS enhancements: integrate CSRF prevention filter in WebHDFS,
|
|
|
+ support OAuth2 in WebHDFS, disallow/allow snapshots via WebHDFS
|
|
|
+
|
|
|
+ * Allow long-running Balancer to login with keytab
|
|
|
+
|
|
|
+ * Add ReverseXML processor which reconstructs an fsimage from an XML file.
|
|
|
+ This will make it easy to create fsimages for testing, and manually edit
|
|
|
+ fsimages when there is corruption
|
|
|
+
|
|
|
+ * Support nested encryption zones
|
|
|
|
|
|
- * Using the OfflineImageViewer, clients can now browse an fsimage via
|
|
|
- the WebHDFS API.
|
|
|
+ * DataNode Lifeline Protocol: an alternative protocol for reporting DataNode
|
|
|
+ liveness. This can prevent the NameNode from incorrectly marking DataNodes
|
|
|
+ as stale or dead in highly overloaded clusters where heartbeat processing
|
|
|
+ is suffering delays.
|
|
|
|
|
|
- * The NFS gateway received a number of supportability improvements and
|
|
|
- bug fixes. The Hadoop portmapper is no longer required to run the gateway,
|
|
|
- and the gateway is now able to reject connections from unprivileged ports.
|
|
|
+ * Logging HDFS operation's caller context into audit logs
|
|
|
|
|
|
- * The SecondaryNameNode, JournalNode, and DataNode web UIs have been
|
|
|
- modernized with HTML5 and Javascript.
|
|
|
+ * A new Datanode command for evicting writers which is useful when data node
|
|
|
+ decommissioning is blocked by slow writers.
|
|
|
|
|
|
* YARN
|
|
|
|
|
|
- * YARN's REST APIs now support write/modify operations. Users can submit and
|
|
|
- kill applications through REST APIs.
|
|
|
+ * NodeManager CPU resource monitoring in Windows.
|
|
|
+
|
|
|
+ * NM shutdown more graceful: NM will unregister to RM immediately rather than
|
|
|
+ waiting for timeout to be LOST (if NM work preserving is not enabled).
|
|
|
+
|
|
|
+ * Add ability to fail a specific AM attempt in scenario of AM attempt get stuck.
|
|
|
+
|
|
|
+ * CallerContext support in YARN audit log.
|
|
|
+
|
|
|
+ * ATS versioning support: a new configuration to indicate timeline service version.
|
|
|
+
|
|
|
+* MAPREDUCE
|
|
|
|
|
|
- * The timeline store in YARN, used for storing generic and
|
|
|
- application-specific information for applications, supports authentication
|
|
|
- through Kerberos.
|
|
|
+ * Allow node labels get specificed in submitting MR jobs
|
|
|
|
|
|
- * The Fair Scheduler supports dynamic hierarchical user queues, user queues
|
|
|
- are created dynamically at runtime under any specified parent-queue.
|
|
|
+ * Add a new tool to combine aggregated logs into HAR files
|
|
|
|
|
|
Getting Started
|
|
|
===============
|
Junping Du