|
|
@@ -1,3 +1,2953 @@
|
|
|
+<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
|
|
|
+<title>Hadoop 2.3.0 Release Notes</title>
|
|
|
+<STYLE type="text/css">
|
|
|
+ H1 {font-family: sans-serif}
|
|
|
+ H2 {font-family: sans-serif; margin-left: 7mm}
|
|
|
+ TABLE {margin-left: 7mm}
|
|
|
+</STYLE>
|
|
|
+</head>
|
|
|
+<body>
|
|
|
+<h1>Hadoop 2.3.0 Release Notes</h1>
|
|
|
+These release notes include new developer and user-facing incompatibilities, features, and major improvements.
|
|
|
+<a name="changes"/>
|
|
|
+<h2>Changes since Hadoop 2.2.0</h2>
|
|
|
+<ul>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1642">YARN-1642</a>.
|
|
|
+ Blocker sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>RMDTRenewer#getRMClient should use ClientRMProxy</b><br>
|
|
|
+ <blockquote>RMDTRenewer#getRMClient gets a proxy to the RM in the conf directly instead of going through ClientRMProxy.
|
|
|
+
|
|
|
+{code}
|
|
|
+ final YarnRPC rpc = YarnRPC.create(conf);
|
|
|
+ return (ApplicationClientProtocol)rpc.getProxy(ApplicationClientProtocol.class, addr, conf);
|
|
|
+{code}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1630">YARN-1630</a>.
|
|
|
+ Major bug reported by Aditya Acharya and fixed by Aditya Acharya (client)<br>
|
|
|
+ <b>Introduce timeout for async polling operations in YarnClientImpl</b><br>
|
|
|
+ <blockquote>I ran an MR2 application that would have been long running, and killed it programmatically using a YarnClient. The app was killed, but the client hung forever. The message that I saw, which spammed the logs, was "Watiting for application application_1389036507624_0018 to be killed."
|
|
|
+
|
|
|
+The RM log indicated that the app had indeed transitioned from RUNNING to KILLED, but for some reason future responses to the RPC to kill the application did not indicate that the app had been terminated.
|
|
|
+
|
|
|
+I tracked this down to YarnClientImpl.java, and though I was unable to reproduce the bug, I wrote a patch to introduce a bound on the number of times that YarnClientImpl retries the RPC before giving up.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1629">YARN-1629</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Sandy Ryza (scheduler)<br>
|
|
|
+ <b>IndexOutOfBoundsException in Fair Scheduler MaxRunningAppsEnforcer</b><br>
|
|
|
+ <blockquote>This can occur when the second-to-last app in a queue's pending app list is made runnable. The app is pulled out from under the iterator. </blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1628">YARN-1628</a>.
|
|
|
+ Major bug reported by Mit Desai and fixed by Vinod Kumar Vavilapalli <br>
|
|
|
+ <b>TestContainerManagerSecurity fails on trunk</b><br>
|
|
|
+ <blockquote>The Test fails with the following error
|
|
|
+
|
|
|
+{noformat}
|
|
|
+java.lang.IllegalArgumentException: java.net.UnknownHostException: InvalidHost
|
|
|
+ at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377)
|
|
|
+ at org.apache.hadoop.yarn.server.security.BaseNMTokenSecretManager.newInstance(BaseNMTokenSecretManager.java:145)
|
|
|
+ at org.apache.hadoop.yarn.server.security.BaseNMTokenSecretManager.createNMToken(BaseNMTokenSecretManager.java:136)
|
|
|
+ at org.apache.hadoop.yarn.server.TestContainerManagerSecurity.testNMTokens(TestContainerManagerSecurity.java:253)
|
|
|
+ at org.apache.hadoop.yarn.server.TestContainerManagerSecurity.testContainerManager(TestContainerManagerSecurity.java:144)
|
|
|
+{noformat}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1624">YARN-1624</a>.
|
|
|
+ Major bug reported by Aditya Acharya and fixed by Aditya Acharya (scheduler)<br>
|
|
|
+ <b>QueuePlacementPolicy format is not easily readable via a JAXB parser</b><br>
|
|
|
+ <blockquote>The current format for specifying queue placement rules in the fair scheduler allocations file does not lend itself to easy parsing via a JAXB parser. In particular, relying on the tag name to encode information about which rule to use makes it very difficult for an xsd-based JAXB parser to preserve the order of the rules, which is essential.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1623">YARN-1623</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Sandy Ryza (scheduler)<br>
|
|
|
+ <b>Include queue name in RegisterApplicationMasterResponse</b><br>
|
|
|
+ <blockquote>This provides the YARN change necessary to support MAPREDUCE-5732.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1618">YARN-1618</a>.
|
|
|
+ Blocker sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>Fix invalid RMApp transition from NEW to FINAL_SAVING</b><br>
|
|
|
+ <blockquote>YARN-891 augments the RMStateStore to store information on completed applications. In the process, it adds transitions from NEW to FINAL_SAVING. This leads to the RM trying to update entries in the state-store that do not exist. On ZKRMStateStore, this leads to the RM crashing.
|
|
|
+
|
|
|
+Previous description:
|
|
|
+ZKRMStateStore fails to handle updates to znodes that don't exist. For instance, this can happen when an app transitions from NEW to FINAL_SAVING. In these cases, the store should create the missing znode and handle the update.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1616">YARN-1616</a>.
|
|
|
+ Trivial improvement reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>RMFatalEventDispatcher should log the cause of the event</b><br>
|
|
|
+ <blockquote>RMFatalEventDispatcher#handle() logs the receipt of an event and its type, but leaves out the cause. The cause captures why the event was raised and would help debugging issues. </blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1608">YARN-1608</a>.
|
|
|
+ Trivial bug reported by Karthik Kambatla and fixed by Karthik Kambatla (nodemanager)<br>
|
|
|
+ <b>LinuxContainerExecutor has a few DEBUG messages at INFO level</b><br>
|
|
|
+ <blockquote>LCE has a few INFO level log messages meant to be at debug level. In fact, they are logged both at INFO and DEBUG. </blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1607">YARN-1607</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Sandy Ryza <br>
|
|
|
+ <b>TestRM expects the capacity scheduler</b><br>
|
|
|
+ <blockquote>We should either explicitly set the Capacity Scheduler or make it scheduler-agnostic</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1603">YARN-1603</a>.
|
|
|
+ Trivial bug reported by Zhijie Shen and fixed by Zhijie Shen <br>
|
|
|
+ <b>Remove two *.orig files which were unexpectedly committed</b><br>
|
|
|
+ <blockquote>FairScheduler.java.orig and TestFifoScheduler.java.orig</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1601">YARN-1601</a>.
|
|
|
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur <br>
|
|
|
+ <b>3rd party JARs are missing from hadoop-dist output</b><br>
|
|
|
+ <blockquote>With the build changes of YARN-888 we are leaving out all 3rd party JArs used directly by YARN under /share/hadoop/yarn/lib/.
|
|
|
+
|
|
|
+We did not notice this when running minicluster because they all happen to be in the classpath from hadoop-common and hadoop-yarn.
|
|
|
+
|
|
|
+As 3d party JARs are not 'public' interfaces we cannot rely on them being provided to yarn by common and hdfs. (ie if common and hdfs stop using a 3rd party dependency that yarn uses this would break yarn if yarn does not pull that dependency explicitly).
|
|
|
+
|
|
|
+Also, this will break bigtop hadoop build when they move to use branch-2 as they expect to find jars in /share/hadoop/yarn/lib/</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1600">YARN-1600</a>.
|
|
|
+ Blocker bug reported by Jason Lowe and fixed by Haohui Mai (resourcemanager)<br>
|
|
|
+ <b>RM does not startup when security is enabled without spnego configured</b><br>
|
|
|
+ <blockquote>We have a custom auth filter in front of our various UI pages that handles user authentication. However currently the RM assumes that if security is enabled then the user must have configured spnego as well for the RM web pages which is not true in our case.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1598">YARN-1598</a>.
|
|
|
+ Critical sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla (client , resourcemanager)<br>
|
|
|
+ <b>HA-related rmadmin commands don't work on a secure cluster</b><br>
|
|
|
+ <blockquote>The HA-related commands like -getServiceState -checkHealth etc. don't work in a secure cluster.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1579">YARN-1579</a>.
|
|
|
+ Trivial sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>ActiveRMInfoProto fields should be optional</b><br>
|
|
|
+ <blockquote>Per discussion on YARN-1568, ActiveRMInfoProto should have optional fields instead of required fields. </blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1575">YARN-1575</a>.
|
|
|
+ Critical sub-task reported by Jason Lowe and fixed by Jason Lowe (nodemanager)<br>
|
|
|
+ <b>Public localizer crashes with "Localized unkown resource"</b><br>
|
|
|
+ <blockquote>The public localizer can crash with the error:
|
|
|
+
|
|
|
+{noformat}
|
|
|
+2014-01-08 14:11:43,212 [Thread-467] ERROR org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Localized unkonwn resource to java.util.concurrent.FutureTask@852e26
|
|
|
+2014-01-08 14:11:43,212 [Thread-467] INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Public cache exiting
|
|
|
+{noformat}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1574">YARN-1574</a>.
|
|
|
+ Blocker sub-task reported by Xuan Gong and fixed by Xuan Gong <br>
|
|
|
+ <b>RMDispatcher should be reset on transition to standby</b><br>
|
|
|
+ <blockquote>Currently, we move rmDispatcher out of ActiveService. But we still register the Event dispatcher, such as schedulerDispatcher, RMAppEventDispatcher when we initiate the ActiveService.
|
|
|
+
|
|
|
+Almost every time when we transit RM from Active to Standby, we need to initiate the ActiveService. That means we will register the same event Dispatcher which will cause the same event will be handled several times.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1573">YARN-1573</a>.
|
|
|
+ Major sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>ZK store should use a private password for root-node-acls</b><br>
|
|
|
+ <blockquote>Currently, when HA is enabled, ZK store uses cluster-timestamp as the password for root node ACLs to give the Active RM exclusive access to the store. A more private value like a random number might be better. </blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1568">YARN-1568</a>.
|
|
|
+ Trivial task reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>Rename clusterid to clusterId in ActiveRMInfoProto </b><br>
|
|
|
+ <blockquote>YARN-1029 introduces ActiveRMInfoProto - just realized it defines a field clusterid, which is inconsistent with other fields. Better to fix it immediately than leave the inconsistency. </blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1567">YARN-1567</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Sandy Ryza (scheduler)<br>
|
|
|
+ <b>In Fair Scheduler, allow empty queues to change between leaf and parent on allocation file reload</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1560">YARN-1560</a>.
|
|
|
+ Major test reported by Ted Yu and fixed by Ted Yu <br>
|
|
|
+ <b>TestYarnClient#testAMMRTokens fails with null AMRM token</b><br>
|
|
|
+ <blockquote>The following can be reproduced locally:
|
|
|
+{code}
|
|
|
+testAMMRTokens(org.apache.hadoop.yarn.client.api.impl.TestYarnClient) Time elapsed: 3.341 sec <<< FAILURE!
|
|
|
+junit.framework.AssertionFailedError: null
|
|
|
+ at junit.framework.Assert.fail(Assert.java:48)
|
|
|
+ at junit.framework.Assert.assertTrue(Assert.java:20)
|
|
|
+ at junit.framework.Assert.assertNotNull(Assert.java:218)
|
|
|
+ at junit.framework.Assert.assertNotNull(Assert.java:211)
|
|
|
+ at org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testAMMRTokens(TestYarnClient.java:382)
|
|
|
+{code}
|
|
|
+This test didn't appear in https://builds.apache.org/job/Hadoop-Yarn-trunk/442/consoleFull</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1559">YARN-1559</a>.
|
|
|
+ Blocker sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>Race between ServerRMProxy and ClientRMProxy setting RMProxy#INSTANCE</b><br>
|
|
|
+ <blockquote>RMProxy#INSTANCE is a non-final static field and both ServerRMProxy and ClientRMProxy set it. This leads to races as witnessed on - YARN-1482.
|
|
|
+
|
|
|
+Sample trace:
|
|
|
+{noformat}
|
|
|
+java.lang.IllegalArgumentException: RM does not support this client protocol
|
|
|
+ at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
|
|
|
+ at org.apache.hadoop.yarn.client.ClientRMProxy.checkAllowedProtocols(ClientRMProxy.java:119)
|
|
|
+ at org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.init(ConfiguredRMFailoverProxyProvider.java:58)
|
|
|
+ at org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:158)
|
|
|
+ at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:88)
|
|
|
+ at org.apache.hadoop.yarn.server.api.ServerRMProxy.createRMProxy(ServerRMProxy.java:56)
|
|
|
+{noformat}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1549">YARN-1549</a>.
|
|
|
+ Major test reported by Ted Yu and fixed by haosdent <br>
|
|
|
+ <b>TestUnmanagedAMLauncher#testDSShell fails in trunk</b><br>
|
|
|
+ <blockquote>The following error is reproducible:
|
|
|
+{code}
|
|
|
+testDSShell(org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher) Time elapsed: 14.911 sec <<< ERROR!
|
|
|
+java.lang.RuntimeException: Failed to receive final expected state in ApplicationReport, CurrentState=RUNNING, ExpectedStates=FINISHED,FAILED,KILLED
|
|
|
+ at org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.monitorApplication(UnmanagedAMLauncher.java:447)
|
|
|
+ at org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.run(UnmanagedAMLauncher.java:352)
|
|
|
+ at org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher.testDSShell(TestUnmanagedAMLauncher.java:147)
|
|
|
+{code}
|
|
|
+See https://builds.apache.org/job/Hadoop-Yarn-trunk/435</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1541">YARN-1541</a>.
|
|
|
+ Major bug reported by Jian He and fixed by Jian He <br>
|
|
|
+ <b>Invalidate AM Host/Port when app attempt is done so that in the mean-while client doesn’t get wrong information.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1527">YARN-1527</a>.
|
|
|
+ Trivial bug reported by Jian He and fixed by Akira AJISAKA <br>
|
|
|
+ <b>yarn rmadmin command prints wrong usage info:</b><br>
|
|
|
+ <blockquote>The usage should be: yarn rmadmin, instead of java RMAdmin, and the -refreshQueues should be in the second line.
|
|
|
+{code} Usage: java RMAdmin -refreshQueues
|
|
|
+ -refreshNodes
|
|
|
+ -refreshSuperUserGroupsConfiguration
|
|
|
+ -refreshUserToGroupsMappings
|
|
|
+ -refreshAdminAcls
|
|
|
+ -refreshServiceAcl
|
|
|
+ -getGroups [username]
|
|
|
+ -help [cmd]
|
|
|
+ -transitionToActive <serviceId>
|
|
|
+ -transitionToStandby <serviceId>
|
|
|
+ -failover [--forcefence] [--forceactive] <serviceId> <serviceId>
|
|
|
+ -getServiceState <serviceId>
|
|
|
+ -checkHealth <serviceId>
|
|
|
+{code}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1523">YARN-1523</a>.
|
|
|
+ Major sub-task reported by Bikas Saha and fixed by Karthik Kambatla <br>
|
|
|
+ <b>Use StandbyException instead of RMNotYetReadyException</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1522">YARN-1522</a>.
|
|
|
+ Major bug reported by Liyin Liang and fixed by Liyin Liang <br>
|
|
|
+ <b>TestApplicationCleanup.testAppCleanup occasionally fails</b><br>
|
|
|
+ <blockquote>TestApplicationCleanup is occasionally failing with the error:
|
|
|
+{code}
|
|
|
+-------------------------------------------------------------------------------
|
|
|
+Test set: org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup
|
|
|
+-------------------------------------------------------------------------------
|
|
|
+Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 6.215 sec <<< FAILURE! - in org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup
|
|
|
+testAppCleanup(org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup) Time elapsed: 5.555 sec <<< FAILURE!
|
|
|
+junit.framework.AssertionFailedError: expected:<1> but was:<0>
|
|
|
+at org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup.testAppCleanup(TestApplicationCleanup.java:119)
|
|
|
+{code}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1505">YARN-1505</a>.
|
|
|
+ Blocker bug reported by Xuan Gong and fixed by Xuan Gong <br>
|
|
|
+ <b>WebAppProxyServer should not set localhost as YarnConfiguration.PROXY_ADDRESS by itself</b><br>
|
|
|
+ <blockquote>At WebAppProxyServer::startServer(), it will set up YarnConfiguration.PROXY_ADDRESS to localhost:9099 by itself. So, no matter what is the value we set YarnConfiguration.PROXY_ADDRESS in configuration, the proxyserver will bind to localhost:9099</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1491">YARN-1491</a>.
|
|
|
+ Trivial bug reported by Jonathan Eagles and fixed by Chen He <br>
|
|
|
+ <b>Upgrade JUnit3 TestCase to JUnit 4</b><br>
|
|
|
+ <blockquote>There are still four references to test classes that extend from junit.framework.TestCase
|
|
|
+
|
|
|
+hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestYarnVersionInfo.java
|
|
|
+hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java
|
|
|
+hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java
|
|
|
+hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1485">YARN-1485</a>.
|
|
|
+ Major sub-task reported by Xuan Gong and fixed by Xuan Gong <br>
|
|
|
+ <b>Enabling HA should verify the RM service addresses configurations have been set for every RM Ids defined in RM_HA_IDs</b><br>
|
|
|
+ <blockquote>After YARN-1325, the YarnConfiguration.RM_HA_IDS will contain multiple RM_Ids. We need to verify that the RM service addresses configurations have been set for all of RM_Ids.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1482">YARN-1482</a>.
|
|
|
+ Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Xuan Gong <br>
|
|
|
+ <b>WebApplicationProxy should be always-on w.r.t HA even if it is embedded in the RM</b><br>
|
|
|
+ <blockquote>This way, even if an RM goes to standby mode, we can affect a redirect to the active. And more importantly, users will not suddenly see all their links stop working.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1481">YARN-1481</a>.
|
|
|
+ Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli <br>
|
|
|
+ <b>Move internal services logic from AdminService to ResourceManager</b><br>
|
|
|
+ <blockquote>This is something I found while reviewing YARN-1318, but didn't halt that patch as many cycles went there already. Some top level issues
|
|
|
+ - Not easy to follow RM's service life cycle
|
|
|
+ -- RM adds only AdminService as its service directly.
|
|
|
+ -- Other services are added to RM when AdminService's init calls RM.activeServices.init()
|
|
|
+ - Overall, AdminService shouldn't encompass all of RM's HA state management. It was originally supposed to be the implementation of just the RPC server.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1463">YARN-1463</a>.
|
|
|
+ Major test reported by Ted Yu and fixed by Vinod Kumar Vavilapalli <br>
|
|
|
+ <b>Tests should avoid starting http-server where possible or creates spnego keytab/principals</b><br>
|
|
|
+ <blockquote>Here is stack trace:
|
|
|
+{code}
|
|
|
+testContainerManager[1](org.apache.hadoop.yarn.server.TestContainerManagerSecurity) Time elapsed: 1.756 sec <<< ERROR!
|
|
|
+org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException: ResourceManager failed to start. Final state is STOPPED
|
|
|
+ at org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceStart(MiniYARNCluster.java:253)
|
|
|
+ at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
|
|
|
+ at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
|
|
|
+ at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
|
|
|
+ at org.apache.hadoop.yarn.server.TestContainerManagerSecurity.testContainerManager(TestContainerManagerSecurity.java:110)
|
|
|
+{code}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1454">YARN-1454</a>.
|
|
|
+ Critical bug reported by Jian He and fixed by Karthik Kambatla <br>
|
|
|
+ <b>TestRMRestart.testRMDelegationTokenRestoredOnRMRestart is failing intermittently </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1451">YARN-1451</a>.
|
|
|
+ Minor bug reported by Sandy Ryza and fixed by Sandy Ryza <br>
|
|
|
+ <b>TestResourceManager relies on the scheduler assigning multiple containers in a single node update</b><br>
|
|
|
+ <blockquote>TestResourceManager rely on the capacity scheduler.
|
|
|
+
|
|
|
+It relies on a scheduler that assigns multiple containers in a single heartbeat, which not all schedulers do by default. It also relies on schedulers that don't consider CPU capacities. It would be simple to change the test to use multiple heartbeats and increase the vcore capacities of the nodes in the test.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1450">YARN-1450</a>.
|
|
|
+ Major bug reported by Akira AJISAKA and fixed by Binglin Chang (applications/distributed-shell)<br>
|
|
|
+ <b>TestUnmanagedAMLauncher#testDSShell fails on trunk</b><br>
|
|
|
+ <blockquote>TestUnmanagedAMLauncher fails on trunk. The console output is
|
|
|
+{code}
|
|
|
+Running org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher
|
|
|
+Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 35.937 sec <<< FAILURE! - in org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher
|
|
|
+testDSShell(org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher) Time elapsed: 14.558 sec <<< ERROR!
|
|
|
+java.lang.RuntimeException: Failed to receive final expected state in ApplicationReport, CurrentState=ACCEPTED, ExpectedStates=FINISHED,FAILED,KILLED
|
|
|
+ at org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.monitorApplication(UnmanagedAMLauncher.java:447)
|
|
|
+ at org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.run(UnmanagedAMLauncher.java:352)
|
|
|
+ at org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher.testDSShell(TestUnmanagedAMLauncher.java:145)
|
|
|
+{code}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1448">YARN-1448</a>.
|
|
|
+ Major sub-task reported by Wangda Tan and fixed by Wangda Tan (api , resourcemanager)<br>
|
|
|
+ <b>AM-RM protocol changes to support container resizing</b><br>
|
|
|
+ <blockquote>As described in YARN-1197, we need add API in RM to support
|
|
|
+1) Add increase request in AllocateRequest
|
|
|
+2) Can get successfully increased/decreased containers from RM in AllocateResponse</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1447">YARN-1447</a>.
|
|
|
+ Major sub-task reported by Wangda Tan and fixed by Wangda Tan (api)<br>
|
|
|
+ <b>Common PB type definitions for container resizing</b><br>
|
|
|
+ <blockquote>As described in YARN-1197, we need add some common PB types for container resource change, like ResourceChangeContext, etc. These types will be both used by RM/NM protocols</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1446">YARN-1446</a>.
|
|
|
+ Major sub-task reported by Jian He and fixed by Jian He (resourcemanager)<br>
|
|
|
+ <b>Change killing application to wait until state store is done</b><br>
|
|
|
+ <blockquote>When user kills an application, it should wait until the state store is done with saving the killed status of the application. Otherwise, if RM crashes in the middle between user killing the application and writing the status to the store, RM will relaunch this application after it restarts.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1435">YARN-1435</a>.
|
|
|
+ Major bug reported by Tassapol Athiapinya and fixed by Xuan Gong (applications/distributed-shell)<br>
|
|
|
+ <b>Distributed Shell should not run other commands except "sh", and run the custom script at the same time.</b><br>
|
|
|
+ <blockquote>Currently, if we want to run custom script at DS. We can do it like this :
|
|
|
+--shell_command sh --shell_script custom_script.sh
|
|
|
+But it may be better to separate running shell_command and shell_script</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1425">YARN-1425</a>.
|
|
|
+ Major bug reported by Omkar Vinit Joshi and fixed by Omkar Vinit Joshi <br>
|
|
|
+ <b>TestRMRestart fails because MockRM.waitForState(AttemptId) uses current attempt instead of the attempt passed as argument</b><br>
|
|
|
+ <blockquote>TestRMRestart is failing on trunk. Fixing it. </blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1423">YARN-1423</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Ted Malaska (scheduler)<br>
|
|
|
+ <b>Support queue placement by secondary group in the Fair Scheduler</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1419">YARN-1419</a>.
|
|
|
+ Minor bug reported by Jonathan Eagles and fixed by Jonathan Eagles (scheduler)<br>
|
|
|
+ <b>TestFifoScheduler.testAppAttemptMetrics fails intermittently under jdk7 </b><br>
|
|
|
+ <blockquote>QueueMetrics holds its data in a static variable causing metrics to bleed over from test to test. clearQueueMetrics is to be called for tests that need to measure metrics correctly for a single test. jdk7 comes into play since tests are run out of order, and in the case make the metrics unreliable.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1416">YARN-1416</a>.
|
|
|
+ Major bug reported by Omkar Vinit Joshi and fixed by Jian He <br>
|
|
|
+ <b>InvalidStateTransitions getting reported in multiple test cases even though they pass</b><br>
|
|
|
+ <blockquote>It might be worth checking why they are reporting this.
|
|
|
+Testcase : TestRMAppTransitions, TestRM
|
|
|
+there are large number of such errors.
|
|
|
+can't handle RMAppEventType.APP_UPDATE_SAVED at RMAppState.FAILED
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1411">YARN-1411</a>.
|
|
|
+ Critical sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla <br>
|
|
|
+ <b>HA config shouldn't affect NodeManager RPC addresses</b><br>
|
|
|
+ <blockquote>When HA is turned on, {{YarnConfiguration#getSoketAddress()}} fetches rpc-addresses corresponding to the specified rm-id. This should only be for RM rpc-addresses. Other confs, like NM rpc-addresses shouldn't be affected by this.
|
|
|
+
|
|
|
+Currently, the NM address settings in yarn-site.xml aren't reflected in the actual ports.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1409">YARN-1409</a>.
|
|
|
+ Major bug reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA <br>
|
|
|
+ <b>NonAggregatingLogHandler can throw RejectedExecutionException</b><br>
|
|
|
+ <blockquote>This problem is caused by handling APPLICATION_FINISHED events after calling sched.shotdown() in NonAggregatingLongHandler#serviceStop(). org.apache.hadoop.mapred.TestJobCleanup can fail because of RejectedExecutionException by NonAggregatingLogHandler.
|
|
|
+
|
|
|
+{code}
|
|
|
+2013-11-13 10:53:06,970 FATAL [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(166)) - Error in dispatcher thread
|
|
|
+java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@d51df63 rejected from java.util.concurrent.ScheduledThreadPoolExecutor@7a20e369[Shutting down, pool size = 4, active threads = 0, queued tasks = 7, completed tasks = 0]
|
|
|
+ at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
|
|
|
+ at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
|
|
|
+ at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
|
|
|
+ at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530)
|
|
|
+ at org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler.handle(NonAggregatingLogHandler.java:121)
|
|
|
+ at org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler.handle(NonAggregatingLogHandler.java:49)
|
|
|
+ at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:159)
|
|
|
+ at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:95)
|
|
|
+ at java.lang.Thread.run(Thread.java:724)
|
|
|
+{code}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1407">YARN-1407</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Sandy Ryza <br>
|
|
|
+ <b>RM Web UI and REST APIs should uniformly use YarnApplicationState</b><br>
|
|
|
+ <blockquote>RMAppState isn't a public facing enum like YarnApplicationState, so we shouldn't return values or list filters that come from it. However, some Blocks and AppInfo are still using RMAppState.
|
|
|
+
|
|
|
+It is not 100% clear to me whether or not fixing this would be a backwards-incompatible change. The change would only reduce the set of possible strings that the API returns, so I think not. We have also been changing the contents of RMAppState since 2.2.0, e.g. in YARN-891. It would still be good to fix this ASAP (i.e. for 2.2.1).</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1405">YARN-1405</a>.
|
|
|
+ Major sub-task reported by Yesha Vora and fixed by Jian He <br>
|
|
|
+ <b>RM hangs on shutdown if calling system.exit in serviceInit or serviceStart</b><br>
|
|
|
+ <blockquote>Enable yarn.resourcemanager.recovery.enabled=true and Pass a local path to yarn.resourcemanager.fs.state-store.uri. such as "file:///tmp/MYTMP"
|
|
|
+
|
|
|
+if the directory /tmp/MYTMP is not readable or writable, RM should crash and should print "Permission denied Error"
|
|
|
+
|
|
|
+Currently, RM throws "java.io.FileNotFoundException: File file:/tmp/MYTMP/FSRMStateRoot/RMDTSecretManagerRoot does not exist" Error. RM returns Exiting status 1 but RM process does not shutdown.
|
|
|
+
|
|
|
+Snapshot of Resource manager log:
|
|
|
+
|
|
|
+2013-09-27 18:31:36,621 INFO security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(97)) - Rolling master-key for nm-tokens
|
|
|
+2013-09-27 18:31:36,694 ERROR resourcemanager.ResourceManager (ResourceManager.java:serviceStart(640)) - Failed to load/recover state
|
|
|
+java.io.FileNotFoundException: File file:/tmp/MYTMP/FSRMStateRoot/RMDTSecretManagerRoot does not exist
|
|
|
+ at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:379)
|
|
|
+ at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1478)
|
|
|
+ at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1518)
|
|
|
+ at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:564)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.loadRMDTSecretManagerState(FileSystemRMStateStore.java:188)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.loadState(FileSystemRMStateStore.java:112)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:635)
|
|
|
+ at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:855)
|
|
|
+2013-09-27 18:31:36,697 INFO util.ExitUtil (ExitUtil.java:terminate(124)) - Exiting with status 1</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1403">YARN-1403</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Sandy Ryza <br>
|
|
|
+ <b>Separate out configuration loading from QueueManager in the Fair Scheduler</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1401">YARN-1401</a>.
|
|
|
+ Major bug reported by Gera Shegalov and fixed by Gera Shegalov (nodemanager)<br>
|
|
|
+ <b>With zero sleep-delay-before-sigkill.ms, no signal is ever sent</b><br>
|
|
|
+ <blockquote>If you set in yarn-site.xml yarn.nodemanager.sleep-delay-before-sigkill.ms=0 then an unresponsive child JVM is never killed. In MRv1, TT used to immediately SIGKILL in this case. </blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1400">YARN-1400</a>.
|
|
|
+ Trivial bug reported by Raja Aluri and fixed by Raja Aluri (resourcemanager)<br>
|
|
|
+ <b>yarn.cmd uses HADOOP_RESOURCEMANAGER_OPTS. Should be YARN_RESOURCEMANAGER_OPTS.</b><br>
|
|
|
+ <blockquote>yarn.cmd uses HADOOP_RESOURCEMANAGER_OPTS. Should be YARN_RESOURCEMANAGER_OPTS.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1395">YARN-1395</a>.
|
|
|
+ Major bug reported by Chris Nauroth and fixed by Chris Nauroth (applications/distributed-shell)<br>
|
|
|
+ <b>Distributed shell application master launched with debug flag can hang waiting for external ls process.</b><br>
|
|
|
+ <blockquote>Distributed shell launched with the debug flag will run {{ApplicationMaster#dumpOutDebugInfo}}. This method launches an external process to run ls and print the contents of the current working directory. We've seen that this can cause the application master to hang on {{Process#waitFor}}.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1392">YARN-1392</a>.
|
|
|
+ Major new feature reported by Sandy Ryza and fixed by Sandy Ryza (scheduler)<br>
|
|
|
+ <b>Allow sophisticated app-to-queue placement policies in the Fair Scheduler</b><br>
|
|
|
+ <blockquote>Currently the Fair Scheduler supports app-to-queue placement by username. It would be beneficial to allow more sophisticated policies that rely on primary and secondary groups and fallbacks.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1388">YARN-1388</a>.
|
|
|
+ Trivial bug reported by Liyin Liang and fixed by Liyin Liang (resourcemanager)<br>
|
|
|
+ <b>Fair Scheduler page always displays blank fair share</b><br>
|
|
|
+ <blockquote>YARN-1044 fixed min/max/used resource display problem in the scheduler page. But the "Fair Share" has the same problem and need to fix it.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1387">YARN-1387</a>.
|
|
|
+ Major improvement reported by Karthik Kambatla and fixed by Karthik Kambatla (api)<br>
|
|
|
+ <b>RMWebServices should use ClientRMService for filtering applications</b><br>
|
|
|
+ <blockquote>YARN's REST API allows filtering applications, this should be moved to ClientRMService to allow Java API also support the same functionality.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1386">YARN-1386</a>.
|
|
|
+ Critical bug reported by Jason Lowe and fixed by Jason Lowe (nodemanager)<br>
|
|
|
+ <b>NodeManager mistakenly loses resources and relocalizes them</b><br>
|
|
|
+ <blockquote>When a local resource that should already be present is requested again, the nodemanager checks to see if it still present. However the method it uses to check for presence is via File.exists() as the user of the nodemanager process. If the resource was a private resource localized for another user, it will be localized to a location that is not accessible by the nodemanager user. Therefore File.exists() returns false, the nodemanager mistakenly believes the resource is no longer available, and it proceeds to localize it over and over.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1381">YARN-1381</a>.
|
|
|
+ Minor bug reported by Ted Yu and fixed by Ted Yu <br>
|
|
|
+ <b>Same relaxLocality appears twice in exception message of AMRMClientImpl#checkLocalityRelaxationConflict() </b><br>
|
|
|
+ <blockquote>Here is related code:
|
|
|
+{code}
|
|
|
+ throw new InvalidContainerRequestException("Cannot submit a "
|
|
|
+ + "ContainerRequest asking for location " + location
|
|
|
+ + " with locality relaxation " + relaxLocality + " when it has "
|
|
|
+ + "already been requested with locality relaxation " + relaxLocality);
|
|
|
+{code}
|
|
|
+The last relaxLocality should be reqs.values().iterator().next().remoteRequest.getRelaxLocality() </blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1378">YARN-1378</a>.
|
|
|
+ Major sub-task reported by Jian He and fixed by Jian He (resourcemanager)<br>
|
|
|
+ <b>Implement a RMStateStore cleaner for deleting application/attempt info</b><br>
|
|
|
+ <blockquote>Now that we are storing the final state of application/attempt instead of removing application/attempt info on application/attempt completion(YARN-891), we need a separate RMStateStore cleaner for cleaning the application/attempt state.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1374">YARN-1374</a>.
|
|
|
+ Blocker bug reported by Devaraj K and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>Resource Manager fails to start due to ConcurrentModificationException</b><br>
|
|
|
+ <blockquote>Resource Manager is failing to start with the below ConcurrentModificationException.
|
|
|
+
|
|
|
+{code:xml}
|
|
|
+2013-10-30 20:22:42,371 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts (include/exclude) list
|
|
|
+2013-10-30 20:22:42,376 INFO org.apache.hadoop.service.AbstractService: Service ResourceManager failed in state INITED; cause: java.util.ConcurrentModificationException
|
|
|
+java.util.ConcurrentModificationException
|
|
|
+ at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
|
|
|
+ at java.util.AbstractList$Itr.next(AbstractList.java:343)
|
|
|
+ at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1010)
|
|
|
+ at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:187)
|
|
|
+ at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:944)
|
|
|
+2013-10-30 20:22:42,378 INFO org.apache.hadoop.yarn.server.resourcemanager.RMHAProtocolService: Transitioning to standby
|
|
|
+2013-10-30 20:22:42,378 INFO org.apache.hadoop.yarn.server.resourcemanager.RMHAProtocolService: Transitioned to standby
|
|
|
+2013-10-30 20:22:42,378 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager
|
|
|
+java.util.ConcurrentModificationException
|
|
|
+ at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
|
|
|
+ at java.util.AbstractList$Itr.next(AbstractList.java:343)
|
|
|
+ at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1010)
|
|
|
+ at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:187)
|
|
|
+ at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:944)
|
|
|
+2013-10-30 20:22:42,379 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: SHUTDOWN_MSG:
|
|
|
+/************************************************************
|
|
|
+SHUTDOWN_MSG: Shutting down ResourceManager at HOST-10-18-40-24/10.18.40.24
|
|
|
+************************************************************/
|
|
|
+{code}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1358">YARN-1358</a>.
|
|
|
+ Minor test reported by Chuan Liu and fixed by Chuan Liu (client)<br>
|
|
|
+ <b>TestYarnCLI fails on Windows due to line endings</b><br>
|
|
|
+ <blockquote>The unit test fails on Windows due to incorrect line endings was used for comparing the output from command line output. Error messages are as follows.
|
|
|
+{noformat}
|
|
|
+junit.framework.ComparisonFailure: expected:<...argument for options[]
|
|
|
+usage: application
|
|
|
+...> but was:<...argument for options[
|
|
|
+]
|
|
|
+usage: application
|
|
|
+...>
|
|
|
+ at junit.framework.Assert.assertEquals(Assert.java:85)
|
|
|
+ at junit.framework.Assert.assertEquals(Assert.java:91)
|
|
|
+ at org.apache.hadoop.yarn.client.cli.TestYarnCLI.testMissingArguments(TestYarnCLI.java:878)
|
|
|
+{noformat}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1357">YARN-1357</a>.
|
|
|
+ Minor test reported by Chuan Liu and fixed by Chuan Liu (nodemanager)<br>
|
|
|
+ <b>TestContainerLaunch.testContainerEnvVariables fails on Windows</b><br>
|
|
|
+ <blockquote>This test fails on Windows due to incorrect use of batch script command. Error messages are as follows.
|
|
|
+{noformat}
|
|
|
+junit.framework.AssertionFailedError: expected:<java.nio.HeapByteBuffer[pos=0 lim=19 cap=19]> but was:<java.nio.HeapByteBuffer[pos=0 lim=19 cap=19]>
|
|
|
+ at junit.framework.Assert.fail(Assert.java:50)
|
|
|
+ at junit.framework.Assert.failNotEquals(Assert.java:287)
|
|
|
+ at junit.framework.Assert.assertEquals(Assert.java:67)
|
|
|
+ at junit.framework.Assert.assertEquals(Assert.java:74)
|
|
|
+ at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testContainerEnvVariables(TestContainerLaunch.java:508)
|
|
|
+{noformat}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1351">YARN-1351</a>.
|
|
|
+ Trivial bug reported by Konstantin Weitz and fixed by Konstantin Weitz (resourcemanager)<br>
|
|
|
+ <b>Invalid string format in Fair Scheduler log warn message</b><br>
|
|
|
+ <blockquote>While trying to print a warning, two values of the wrong type (Resource instead of int) are passed into a String.format method call, leading to a runtime exception, in the file:
|
|
|
+
|
|
|
+_trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java_.
|
|
|
+
|
|
|
+The warning was intended to be printed whenever the resources don't fit into each other, either because the number of virtual cores or the memory is too small. I changed the %d's into %s, this way the warning will contain both the cores and the memory.
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1349">YARN-1349</a>.
|
|
|
+ Major bug reported by Chris Nauroth and fixed by Chris Nauroth (client)<br>
|
|
|
+ <b>yarn.cmd does not support passthrough to any arbitrary class.</b><br>
|
|
|
+ <blockquote>The yarn shell script supports passthrough to calling any arbitrary class if the first argument is not one of the per-defined sub-commands. The equivalent cmd script does not implement this and instead fails trying to do a labeled goto to the first argument.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1343">YARN-1343</a>.
|
|
|
+ Critical bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (resourcemanager)<br>
|
|
|
+ <b>NodeManagers additions/restarts are not reported as node updates in AllocateResponse responses to AMs</b><br>
|
|
|
+ <blockquote>If a NodeManager joins the cluster or gets restarted, running AMs never receive the node update indicating the Node is running.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1335">YARN-1335</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Sandy Ryza (scheduler)<br>
|
|
|
+ <b>Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into SchedulerApplication</b><br>
|
|
|
+ <blockquote>FSSchedulerApp and FiCaSchedulerApp use duplicate code in a lot of places. They both extend SchedulerApplication. We can move a lot of this duplicate code into SchedulerApplication.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1333">YARN-1333</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Tsuyoshi OZAWA (scheduler)<br>
|
|
|
+ <b>Support blacklisting in the Fair Scheduler</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1332">YARN-1332</a>.
|
|
|
+ Minor improvement reported by Sandy Ryza and fixed by Sebastian Wong <br>
|
|
|
+ <b>In TestAMRMClient, replace assertTrue with assertEquals where possible</b><br>
|
|
|
+ <blockquote>TestAMRMClient uses a lot of "assertTrue(amClient.ask.size() == 0)" where "assertEquals(0, amClient.ask.size())" would make it easier to see why it's failing at a glance.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1331">YARN-1331</a>.
|
|
|
+ Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (client)<br>
|
|
|
+ <b>yarn.cmd exits with NoClassDefFoundError trying to run rmadmin or logs</b><br>
|
|
|
+ <blockquote>The yarn shell script was updated so that the rmadmin and logs sub-commands launch {{org.apache.hadoop.yarn.client.cli.RMAdminCLI}} and {{org.apache.hadoop.yarn.client.cli.LogsCLI}}. The yarn.cmd script also needs to be updated so that the commands work on Windows.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1325">YARN-1325</a>.
|
|
|
+ Major sub-task reported by Tsuyoshi OZAWA and fixed by Xuan Gong (resourcemanager)<br>
|
|
|
+ <b>Enabling HA should check Configuration contains multiple RMs</b><br>
|
|
|
+ <blockquote>Currently, we can enable RM HA configuration without multiple RM ids(YarnConfiguration.RM_HA_IDS). This behaviour can cause wrong operations. ResourceManager should verify that more than 1 RM id must be specified in RM-HA-IDs.
|
|
|
+
|
|
|
+One idea is to support "strict mode" to enforce this check as configuration(e.g. yarn.resourcemanager.ha.strict-mode.enabled).</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1323">YARN-1323</a>.
|
|
|
+ Major sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla <br>
|
|
|
+ <b>Set HTTPS webapp address along with other RPC addresses in HAUtil</b><br>
|
|
|
+ <blockquote>YARN-1232 adds the ability to configure multiple RMs, but missed out the https web app address. Need to add that in.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1321">YARN-1321</a>.
|
|
|
+ Blocker bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (client)<br>
|
|
|
+ <b>NMTokenCache is a singleton, prevents multiple AMs running in a single JVM to work correctly</b><br>
|
|
|
+ <blockquote>NMTokenCache is a singleton. Because of this, if running multiple AMs in a single JVM NMTokens for the same node from different AMs step on each other and starting containers fail due to mismatch tokens.
|
|
|
+
|
|
|
+The error observed in the client side is something like:
|
|
|
+
|
|
|
+{code}
|
|
|
+ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:llama (auth:PROXY) via llama (auth:SIMPLE) cause:org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.
|
|
|
+NMToken for application attempt : appattempt_1382038445650_0002_000001 was used for starting container with container token issued for application attempt : appattempt_1382038445650_0001_000001
|
|
|
+{code}
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1320">YARN-1320</a>.
|
|
|
+ Major bug reported by Tassapol Athiapinya and fixed by Xuan Gong (applications/distributed-shell)<br>
|
|
|
+ <b>Custom log4j properties in Distributed shell does not work properly.</b><br>
|
|
|
+ <blockquote>Distributed shell cannot pick up custom log4j properties (specified with -log_properties). It always uses default log4j properties.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1318">YARN-1318</a>.
|
|
|
+ Blocker sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>Promote AdminService to an Always-On service and merge in RMHAProtocolService</b><br>
|
|
|
+ <blockquote>Per discussion in YARN-1068, we want AdminService to handle HA-admin operations in addition to the regular non-HA admin operations. To facilitate this, we need to move AdminService an Always-On service. </blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1315">YARN-1315</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Sandy Ryza (resourcemanager , scheduler)<br>
|
|
|
+ <b>TestQueueACLs should also test FairScheduler</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1314">YARN-1314</a>.
|
|
|
+ Major bug reported by Tassapol Athiapinya and fixed by Xuan Gong (applications/distributed-shell)<br>
|
|
|
+ <b>Cannot pass more than 1 argument to shell command</b><br>
|
|
|
+ <blockquote>Distributed shell cannot accept more than 1 parameters in argument parts.
|
|
|
+
|
|
|
+All of these commands are treated as 1 parameter:
|
|
|
+
|
|
|
+/usr/bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client -jar <distrubuted shell jar> -shell_command echo -shell_args "'"My name" "is Teddy"'"
|
|
|
+/usr/bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client -jar <distrubuted shell jar> -shell_command echo -shell_args "''My name' 'is Teddy''"
|
|
|
+/usr/bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client -jar <distrubuted shell jar> -shell_command echo -shell_args "'My name' 'is Teddy'"</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1311">YARN-1311</a>.
|
|
|
+ Trivial sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli <br>
|
|
|
+ <b>Fix app specific scheduler-events' names to be app-attempt based</b><br>
|
|
|
+ <blockquote>Today, APP_ADDED and APP_REMOVED are sent to the scheduler. They are misnomers as schedulers only deal with AppAttempts today. This JIRA is for fixing their names so that we can add App-level events in the near future, notably for work-preserving RM-restart.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1307">YARN-1307</a>.
|
|
|
+ Major sub-task reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA (resourcemanager)<br>
|
|
|
+ <b>Rethink znode structure for RM HA</b><br>
|
|
|
+ <blockquote>Rethink for znode structure for RM HA is proposed in some JIRAs(YARN-659, YARN-1222). The motivation of this JIRA is quoted from Bikas' comment in YARN-1222:
|
|
|
+{quote}
|
|
|
+We should move to creating a node hierarchy for apps such that all znodes for an app are stored under an app znode instead of the app root znode. This will help in removeApplication and also in scaling better on ZK. The earlier code was written this way to ensure create/delete happens under a root znode for fencing. But given that we have moved to multi-operations globally, this isnt required anymore.
|
|
|
+{quote}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1306">YARN-1306</a>.
|
|
|
+ Major bug reported by Wei Yan and fixed by Wei Yan <br>
|
|
|
+ <b>Clean up hadoop-sls sample-conf according to YARN-1228</b><br>
|
|
|
+ <blockquote>Move fair scheduler allocations configuration to fair-scheduler.xml, and move all scheduler stuffs to yarn-site.xml</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1305">YARN-1305</a>.
|
|
|
+ Major sub-task reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA (resourcemanager)<br>
|
|
|
+ <b>RMHAProtocolService#serviceInit should handle HAUtil's IllegalArgumentException</b><br>
|
|
|
+ <blockquote>When yarn.resourcemanager.ha.enabled is true, RMHAProtocolService#serviceInit calls HAUtil.setAllRpcAddresses. If the configuration values are null, it just throws IllegalArgumentException.
|
|
|
+It's messy to analyse which keys are null, so we should handle it and log the name of keys which are null.
|
|
|
+
|
|
|
+A current log dump is as follows:
|
|
|
+{code}
|
|
|
+2013-10-15 06:24:53,431 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: registered UNIX signal handlers for [TERM, HUP, INT]
|
|
|
+2013-10-15 06:24:54,203 INFO org.apache.hadoop.service.AbstractService: Service RMHAProtocolService failed in state INITED; cause: java.lang.IllegalArgumentException: Property value must not be null
|
|
|
+java.lang.IllegalArgumentException: Property value must not be null
|
|
|
+ at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
|
|
|
+ at org.apache.hadoop.conf.Configuration.set(Configuration.java:816)
|
|
|
+ at org.apache.hadoop.conf.Configuration.set(Configuration.java:798)
|
|
|
+ at org.apache.hadoop.yarn.conf.HAUtil.setConfValue(HAUtil.java:100)
|
|
|
+ at org.apache.hadoop.yarn.conf.HAUtil.setAllRpcAddresses(HAUtil.java:105)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.RMHAProtocolService.serviceInit(RMHAProtocolService.java:60)
|
|
|
+ at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
|
|
|
+ at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:187)
|
|
|
+ at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:940)
|
|
|
+{code}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1303">YARN-1303</a>.
|
|
|
+ Major improvement reported by Tassapol Athiapinya and fixed by Xuan Gong (applications/distributed-shell)<br>
|
|
|
+ <b>Allow multiple commands separating with ";" in distributed-shell</b><br>
|
|
|
+ <blockquote>In shell, we can do "ls; ls" to run 2 commands at once.
|
|
|
+
|
|
|
+In distributed shell, this is not working. We should improve to allow this to occur. There are practical use cases that I know of to run multiple commands or to set environment variables before a command.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1300">YARN-1300</a>.
|
|
|
+ Major bug reported by Ted Yu and fixed by Ted Yu <br>
|
|
|
+ <b>SLS tests fail because conf puts yarn properties in fair-scheduler.xml</b><br>
|
|
|
+ <blockquote>I was looking at https://builds.apache.org/job/PreCommit-YARN-Build/2165//testReport/org.apache.hadoop.yarn.sls/TestSLSRunner/testSimulatorRunning/
|
|
|
+I am able to reproduce the failure locally.
|
|
|
+
|
|
|
+I found that FairSchedulerConfiguration.getAllocationFile() doesn't read the yarn.scheduler.fair.allocation.file config entry from fair-scheduler.xml
|
|
|
+
|
|
|
+This leads to the following:
|
|
|
+{code}
|
|
|
+Caused by: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationConfigurationException: Bad fair scheduler config file: top-level element not <allocations>
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.reloadAllocs(QueueManager.java:302)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.initialize(QueueManager.java:108)
|
|
|
+ at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.reinitialize(FairScheduler.java:1145)
|
|
|
+{code}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1295">YARN-1295</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Sandy Ryza (nodemanager)<br>
|
|
|
+ <b>In UnixLocalWrapperScriptBuilder, using bash -c can cause "Text file busy" errors</b><br>
|
|
|
+ <blockquote>I missed this when working on YARN-1271.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1293">YARN-1293</a>.
|
|
|
+ Major bug reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA <br>
|
|
|
+ <b>TestContainerLaunch.testInvalidEnvSyntaxDiagnostics fails on trunk</b><br>
|
|
|
+ <blockquote>{quote}
|
|
|
+-------------------------------------------------------------------------------
|
|
|
+Test set: org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch
|
|
|
+-------------------------------------------------------------------------------
|
|
|
+Tests run: 8, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 12.655 sec <<< FAILURE! - in org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch
|
|
|
+testInvalidEnvSyntaxDiagnostics(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch) Time elapsed: 0.114 sec <<< FAILURE!
|
|
|
+junit.framework.AssertionFailedError: null
|
|
|
+ at junit.framework.Assert.fail(Assert.java:48)
|
|
|
+ at junit.framework.Assert.assertTrue(Assert.java:20)
|
|
|
+ at junit.framework.Assert.assertTrue(Assert.java:27)
|
|
|
+ at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testInvalidEnvSyntaxDiagnostics(TestContainerLaunch.java:273)
|
|
|
+{quote}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1290">YARN-1290</a>.
|
|
|
+ Major improvement reported by Wei Yan and fixed by Wei Yan <br>
|
|
|
+ <b>Let continuous scheduling achieve more balanced task assignment</b><br>
|
|
|
+ <blockquote>Currently, in continuous scheduling (YARN-1010), in each round, the thread iterates over pre-ordered nodes and assigns tasks. This mechanism may overload the first several nodes, while the latter nodes have no tasks.
|
|
|
+
|
|
|
+We should sort all nodes according to available resource. In each round, always assign tasks to nodes with larger capacity, which can balance the load distribution among all nodes.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1288">YARN-1288</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Sandy Ryza (scheduler)<br>
|
|
|
+ <b>Make Fair Scheduler ACLs more user friendly</b><br>
|
|
|
+ <blockquote>The Fair Scheduler currently defaults the root queue's acl to empty and all other queues' acl to "*". Now that YARN-1258 enables configuring the root queue, we should reverse this. This will also bring the Fair Scheduler in line with the Capacity Scheduler.
|
|
|
+
|
|
|
+We should also not trim the acl strings, which makes it impossible to only specify groups in an acl.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1284">YARN-1284</a>.
|
|
|
+ Blocker bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (nodemanager)<br>
|
|
|
+ <b>LCE: Race condition leaves dangling cgroups entries for killed containers</b><br>
|
|
|
+ <blockquote>When LCE & cgroups are enabled, when a container is is killed (in this case by its owning AM, an MRAM) it seems to be a race condition at OS level when doing a SIGTERM/SIGKILL and when the OS does all necessary cleanup.
|
|
|
+
|
|
|
+LCE code, after sending the SIGTERM/SIGKILL and getting the exitcode, immediately attempts to clean up the cgroups entry for the container. But this is failing with an error like:
|
|
|
+
|
|
|
+{code}
|
|
|
+2013-10-07 15:21:24,359 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container container_1381179532433_0016_01_000011 is : 143
|
|
|
+2013-10-07 15:21:24,359 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Processing container_1381179532433_0016_01_000011 of type UPDATE_DIAGNOSTICS_MSG
|
|
|
+2013-10-07 15:21:24,359 DEBUG org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler: deleteCgroup: /run/cgroups/cpu/hadoop-yarn/container_1381179532433_0016_01_000011
|
|
|
+2013-10-07 15:21:24,359 WARN org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler: Unable to delete cgroup at: /run/cgroups/cpu/hadoop-yarn/container_1381179532433_0016_01_000011
|
|
|
+{code}
|
|
|
+
|
|
|
+
|
|
|
+CgroupsLCEResourcesHandler.clearLimits() has logic to wait for 500 ms for AM containers to avoid this problem. it seems this should be done for all containers.
|
|
|
+
|
|
|
+Still, waiting for extra 500ms seems too expensive.
|
|
|
+
|
|
|
+We should look at a way of doing this in a more 'efficient way' from time perspective, may be spinning while the deleteCgroup() cannot be done with a minimal sleep and a timeout.
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1283">YARN-1283</a>.
|
|
|
+ Major sub-task reported by Yesha Vora and fixed by Omkar Vinit Joshi <br>
|
|
|
+ <b>Invalid 'url of job' mentioned in Job output with yarn.http.policy=HTTPS_ONLY</b><br>
|
|
|
+ <blockquote>After setting yarn.http.policy=HTTPS_ONLY, the job output shows incorrect "The url to track the job".
|
|
|
+
|
|
|
+Currently, its printing http://RM:<httpsport>/proxy/application_1381162886563_0001/ instead https://RM:<httpsport>/proxy/application_1381162886563_0001/
|
|
|
+
|
|
|
+http://hostname:8088/proxy/application_1381162886563_0001/ is invalid
|
|
|
+
|
|
|
+hadoop jar hadoop-mapreduce-client-jobclient-tests.jar sleep -m 1 -r 1
|
|
|
+13/10/07 18:39:39 INFO client.RMProxy: Connecting to ResourceManager at hostname/100.00.00.000:8032
|
|
|
+13/10/07 18:39:40 INFO mapreduce.JobSubmitter: number of splits:1
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapreduce.partitioner.class is deprecated. Instead, use mapreduce.job.partitioner.class
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
|
|
|
+13/10/07 18:39:40 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
|
|
|
+13/10/07 18:39:40 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1381162886563_0001
|
|
|
+13/10/07 18:39:40 INFO impl.YarnClientImpl: Submitted application application_1381162886563_0001 to ResourceManager at hostname/100.00.00.000:8032
|
|
|
+13/10/07 18:39:40 INFO mapreduce.Job: The url to track the job: http://hostname:8088/proxy/application_1381162886563_0001/
|
|
|
+13/10/07 18:39:40 INFO mapreduce.Job: Running job: job_1381162886563_0001
|
|
|
+13/10/07 18:39:46 INFO mapreduce.Job: Job job_1381162886563_0001 running in uber mode : false
|
|
|
+13/10/07 18:39:46 INFO mapreduce.Job: map 0% reduce 0%
|
|
|
+13/10/07 18:39:53 INFO mapreduce.Job: map 100% reduce 0%
|
|
|
+13/10/07 18:39:58 INFO mapreduce.Job: map 100% reduce 100%
|
|
|
+13/10/07 18:39:58 INFO mapreduce.Job: Job job_1381162886563_0001 completed successfully
|
|
|
+13/10/07 18:39:58 INFO mapreduce.Job: Counters: 43
|
|
|
+ File System Counters
|
|
|
+ FILE: Number of bytes read=26
|
|
|
+ FILE: Number of bytes written=177279
|
|
|
+ FILE: Number of read operations=0
|
|
|
+ FILE: Number of large read operations=0
|
|
|
+ FILE: Number of write operations=0
|
|
|
+ HDFS: Number of bytes read=48
|
|
|
+ HDFS: Number of bytes written=0
|
|
|
+ HDFS: Number of read operations=1
|
|
|
+ HDFS: Number of large read operations=0
|
|
|
+ HDFS: Number of write operations=0
|
|
|
+ Job Counters
|
|
|
+ Launched map tasks=1
|
|
|
+ Launched reduce tasks=1
|
|
|
+ Other local map tasks=1
|
|
|
+ Total time spent by all maps in occupied slots (ms)=7136
|
|
|
+ Total time spent by all reduces in occupied slots (ms)=6062
|
|
|
+ Map-Reduce Framework
|
|
|
+ Map input records=1
|
|
|
+ Map output records=1
|
|
|
+ Map output bytes=4
|
|
|
+ Map output materialized bytes=22
|
|
|
+ Input split bytes=48
|
|
|
+ Combine input records=0
|
|
|
+ Combine output records=0
|
|
|
+ Reduce input groups=1
|
|
|
+ Reduce shuffle bytes=22
|
|
|
+ Reduce input records=1
|
|
|
+ Reduce output records=0
|
|
|
+ Spilled Records=2
|
|
|
+ Shuffled Maps =1
|
|
|
+ Failed Shuffles=0
|
|
|
+ Merged Map outputs=1
|
|
|
+ GC time elapsed (ms)=60
|
|
|
+ CPU time spent (ms)=1700
|
|
|
+ Physical memory (bytes) snapshot=567582720
|
|
|
+ Virtual memory (bytes) snapshot=4292997120
|
|
|
+ Total committed heap usage (bytes)=846594048
|
|
|
+ Shuffle Errors
|
|
|
+ BAD_ID=0
|
|
|
+ CONNECTION=0
|
|
|
+ IO_ERROR=0
|
|
|
+ WRONG_LENGTH=0
|
|
|
+ WRONG_MAP=0
|
|
|
+ WRONG_REDUCE=0
|
|
|
+ File Input Format Counters
|
|
|
+ Bytes Read=0
|
|
|
+ File Output Format Counters
|
|
|
+ Bytes Written=0
|
|
|
+
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1268">YARN-1268</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Sandy Ryza (scheduler)<br>
|
|
|
+ <b>TestFairScheduler.testContinuousScheduling is flaky</b><br>
|
|
|
+ <blockquote>It looks like there's a timeout in it that's causing it to be flaky.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1265">YARN-1265</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Sandy Ryza (resourcemanager , scheduler)<br>
|
|
|
+ <b>Fair Scheduler chokes on unhealthy node reconnect</b><br>
|
|
|
+ <blockquote>Only nodes in the RUNNING state are tracked by schedulers. When a node reconnects, RMNodeImpl.ReconnectNodeTransition tries to remove it, even if it's in the RUNNING state. The FairScheduler doesn't guard against this.
|
|
|
+
|
|
|
+I think the best way to fix this is to check to see whether a node is RUNNING before telling the scheduler to remove it.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1259">YARN-1259</a>.
|
|
|
+ Trivial bug reported by Sandy Ryza and fixed by Robert Kanter (scheduler)<br>
|
|
|
+ <b>In Fair Scheduler web UI, queue num pending and num active apps switched</b><br>
|
|
|
+ <blockquote>The values returned in FairSchedulerLeafQueueInfo by numPendingApplications and numActiveApplications should be switched.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1258">YARN-1258</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Sandy Ryza (scheduler)<br>
|
|
|
+ <b>Allow configuring the Fair Scheduler root queue</b><br>
|
|
|
+ <blockquote>This would be useful for acls, maxRunningApps, scheduling modes, etc.
|
|
|
+
|
|
|
+The allocation file should be able to accept both:
|
|
|
+* An implicit root queue
|
|
|
+* A root queue at the top of the hierarchy with all queues under/inside of it</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1253">YARN-1253</a>.
|
|
|
+ Blocker new feature reported by Alejandro Abdelnur and fixed by Roman Shaposhnik (nodemanager)<br>
|
|
|
+ <b>Changes to LinuxContainerExecutor to run containers as a single dedicated user in non-secure mode</b><br>
|
|
|
+ <blockquote>When using cgroups we require LCE to be configured in the cluster to start containers.
|
|
|
+
|
|
|
+When LCE starts containers as the user that submitted the job. While this works correctly in a secure setup, in an un-secure setup this presents a couple issues:
|
|
|
+
|
|
|
+* LCE requires all Hadoop users submitting jobs to be Unix users in all nodes
|
|
|
+* Because users can impersonate other users, any user would have access to any local file of other users
|
|
|
+
|
|
|
+Particularly, the second issue is not desirable as a user could get access to ssh keys of other users in the nodes or if there are NFS mounts, get to other users data outside of the cluster.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1241">YARN-1241</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Sandy Ryza <br>
|
|
|
+ <b>In Fair Scheduler, maxRunningApps does not work for non-leaf queues</b><br>
|
|
|
+ <blockquote>Setting the maxRunningApps property on a parent queue should make it that the sum of apps in all subqueues can't exceed it</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1239">YARN-1239</a>.
|
|
|
+ Major sub-task reported by Bikas Saha and fixed by Jian He (resourcemanager)<br>
|
|
|
+ <b>Save version information in the state store</b><br>
|
|
|
+ <blockquote>When creating root dir for the first time we should write version 1. If root dir exists then we should check that the version in the state store matches the version from config.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1232">YARN-1232</a>.
|
|
|
+ Major sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>Configuration to support multiple RMs</b><br>
|
|
|
+ <blockquote>We should augment the configuration to allow users specify two RMs and the individual RPC addresses for them.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1222">YARN-1222</a>.
|
|
|
+ Major sub-task reported by Bikas Saha and fixed by Karthik Kambatla <br>
|
|
|
+ <b>Make improvements in ZKRMStateStore for fencing</b><br>
|
|
|
+ <blockquote>Using multi-operations for every ZK interaction.
|
|
|
+In every operation, automatically creating/deleting a lock znode that is the child of the root znode. This is to achieve fencing by modifying the create/delete permissions on the root znode.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1210">YARN-1210</a>.
|
|
|
+ Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Omkar Vinit Joshi <br>
|
|
|
+ <b>During RM restart, RM should start a new attempt only when previous attempt exits for real</b><br>
|
|
|
+ <blockquote>When RM recovers, it can wait for existing AMs to contact RM back and then kill them forcefully before even starting a new AM. Worst case, RM will start a new AppAttempt after waiting for 10 mins ( the expiry interval). This way we'll minimize multiple AMs racing with each other. This can help issues with downstream components like Pig, Hive and Oozie during RM restart.
|
|
|
+
|
|
|
+In the mean while, new apps will proceed as usual as existing apps wait for recovery.
|
|
|
+
|
|
|
+This can continue to be useful after work-preserving restart, so that AMs which can properly sync back up with RM can continue to run and those that don't are guaranteed to be killed before starting a new attempt.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1199">YARN-1199</a>.
|
|
|
+ Major improvement reported by Mit Desai and fixed by Mit Desai <br>
|
|
|
+ <b>Make NM/RM Versions Available</b><br>
|
|
|
+ <blockquote>Now as we have the NM and RM Versions available, we can display the YARN version of nodes running in the cluster.
|
|
|
+
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1188">YARN-1188</a>.
|
|
|
+ Trivial bug reported by Akira AJISAKA and fixed by Tsuyoshi OZAWA <br>
|
|
|
+ <b>The context of QueueMetrics becomes 'default' when using FairScheduler</b><br>
|
|
|
+ <blockquote>I found the context of QueueMetrics changed to 'default' from 'yarn' when I was using FairScheduler.
|
|
|
+The context should always be 'yarn' by adding an annotation to FSQueueMetrics like below:
|
|
|
+
|
|
|
+{code}
|
|
|
++ @Metrics(context="yarn")
|
|
|
+public class FSQueueMetrics extends QueueMetrics {
|
|
|
+{code}</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1185">YARN-1185</a>.
|
|
|
+ Major sub-task reported by Jason Lowe and fixed by Omkar Vinit Joshi (resourcemanager)<br>
|
|
|
+ <b>FileSystemRMStateStore can leave partial files that prevent subsequent recovery</b><br>
|
|
|
+ <blockquote>FileSystemRMStateStore writes directly to the destination file when storing state. However if the RM were to crash in the middle of the write, the recovery method could encounter a partially-written file and either outright crash during recovery or silently load incomplete state.
|
|
|
+
|
|
|
+To avoid this, the data should be written to a temporary file and renamed to the destination file afterwards.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1183">YARN-1183</a>.
|
|
|
+ Major bug reported by Andrey Klochkov and fixed by Andrey Klochkov <br>
|
|
|
+ <b>MiniYARNCluster shutdown takes several minutes intermittently</b><br>
|
|
|
+ <blockquote>As described in MAPREDUCE-5501 sometimes M/R tests leave MRAppMaster java processes living for several minutes after successful completion of the corresponding test. There is a concurrency issue in MiniYARNCluster shutdown logic which leads to this. Sometimes RM stops before an app master sends it's last report, and then the app master keeps retrying for >6 minutes. In some cases it leads to failures in subsequent tests, and it affects performance of tests as app masters eat resources.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1182">YARN-1182</a>.
|
|
|
+ Major bug reported by Karthik Kambatla and fixed by Karthik Kambatla <br>
|
|
|
+ <b>MiniYARNCluster creates and inits the RM/NM only on start()</b><br>
|
|
|
+ <blockquote>MiniYARNCluster creates and inits the RM/NM only on start(). It should create and init() during init() itself.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1181">YARN-1181</a>.
|
|
|
+ Major sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla <br>
|
|
|
+ <b>Augment MiniYARNCluster to support HA mode</b><br>
|
|
|
+ <blockquote>MiniYARNHACluster, along the lines of MiniYARNCluster, is needed for end-to-end HA tests.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1180">YARN-1180</a>.
|
|
|
+ Trivial bug reported by Thomas Graves and fixed by Chen He (capacityscheduler)<br>
|
|
|
+ <b>Update capacity scheduler docs to include types on the configs</b><br>
|
|
|
+ <blockquote>The capacity scheduler docs (http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html) don't include types for all the configs. For instance the minimum-user-limit-percent doesn't say its an Int. It also the only setting for the Resource Allocation configs that is an Int rather then a float.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1176">YARN-1176</a>.
|
|
|
+ Critical bug reported by Thomas Graves and fixed by Jonathan Eagles (resourcemanager)<br>
|
|
|
+ <b>RM web services ClusterMetricsInfo total nodes doesn't include unhealthy nodes</b><br>
|
|
|
+ <blockquote>In the web services api for the cluster/metrics, the totalNodes reported doesn't include the unhealthy nodes.
|
|
|
+
|
|
|
+this.totalNodes = activeNodes + lostNodes + decommissionedNodes
|
|
|
+ + rebootedNodes;</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1172">YARN-1172</a>.
|
|
|
+ Major sub-task reported by Karthik Kambatla and fixed by Tsuyoshi OZAWA (resourcemanager)<br>
|
|
|
+ <b>Convert *SecretManagers in the RM to services</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1145">YARN-1145</a>.
|
|
|
+ Major bug reported by Rohith and fixed by Rohith <br>
|
|
|
+ <b>Potential file handle leak in aggregated logs web ui</b><br>
|
|
|
+ <blockquote>Any problem in getting aggregated logs for rendering on web ui, then LogReader is not closed.
|
|
|
+
|
|
|
+Now, it reader is not closed which causing many connections in close_wait state.
|
|
|
+
|
|
|
+hadoopuser@hadoopuser:> jps
|
|
|
+*27909* JobHistoryServer
|
|
|
+
|
|
|
+DataNode port is 50010. When greped with DataNode port, many connections are in CLOSE_WAIT from JHS.
|
|
|
+hadoopuser@hadoopuser:> netstat -tanlp |grep 50010
|
|
|
+tcp 0 0 10.18.40.48:50010 0.0.0.0:* LISTEN 21453/java
|
|
|
+tcp 1 0 10.18.40.48:20596 10.18.40.48:50010 CLOSE_WAIT *27909*/java
|
|
|
+tcp 1 0 10.18.40.48:19667 10.18.40.152:50010 CLOSE_WAIT *27909*/java
|
|
|
+tcp 1 0 10.18.40.48:20593 10.18.40.48:50010 CLOSE_WAIT *27909*/java
|
|
|
+tcp 1 0 10.18.40.48:12290 10.18.40.48:50010 CLOSE_WAIT *27909*/java
|
|
|
+tcp 1 0 10.18.40.48:19662 10.18.40.152:50010 CLOSE_WAIT *27909*/java </blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1138">YARN-1138</a>.
|
|
|
+ Major bug reported by Yingda Chen and fixed by Chuan Liu (api)<br>
|
|
|
+ <b>yarn.application.classpath is set to point to $HADOOP_CONF_DIR etc., which does not work on Windows</b><br>
|
|
|
+ <blockquote>yarn-default.xml has "yarn.application.classpath" entry set to $HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/share/hadoop/common/,$HADOOP_COMMON_HOME/share/hadoop/common/lib/,$HADOOP_HDFS_HOME/share/hadoop/hdfs/,$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/,$HADOOP_YARN_HOME/share/hadoop/yarn/*,$HADOOP_YARN_HOME/share/hadoop/yarn/lib. It does not work on Windows which needs to be fixed.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1121">YARN-1121</a>.
|
|
|
+ Major sub-task reported by Bikas Saha and fixed by Jian He (resourcemanager)<br>
|
|
|
+ <b>RMStateStore should flush all pending store events before closing</b><br>
|
|
|
+ <blockquote>on serviceStop it should wait for all internal pending events to drain before stopping.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1119">YARN-1119</a>.
|
|
|
+ Major test reported by Robert Parker and fixed by Mit Desai (resourcemanager)<br>
|
|
|
+ <b>Add ClusterMetrics checks to tho TestRMNodeTransitions tests</b><br>
|
|
|
+ <blockquote>YARN-1101 identified an issue where UNHEALTHY nodes could double decrement the active nodes. We should add checks for RUNNING node transitions.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1109">YARN-1109</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by haosdent (nodemanager)<br>
|
|
|
+ <b>Demote NodeManager "Sending out status for container" logs to debug</b><br>
|
|
|
+ <blockquote>Diagnosing NodeManager and container launch problems is made more difficult by the enormous number of logs like
|
|
|
+{code}
|
|
|
+Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 18, cluster_timestamp: 1377559361179, }, attemptId: 1, }, id: 1337, }, state: C_RUNNING, diagnostics: "Container killed by the ApplicationMaster.\n", exit_status: -1000
|
|
|
+{code}
|
|
|
+
|
|
|
+On an NM with a few containers I am seeing tens of these per second.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1101">YARN-1101</a>.
|
|
|
+ Major bug reported by Robert Parker and fixed by Robert Parker (resourcemanager)<br>
|
|
|
+ <b>Active nodes can be decremented below 0</b><br>
|
|
|
+ <blockquote>The issue is in RMNodeImpl where both RUNNING and UNHEALTHY states that transition to a deactive state (LOST, DECOMMISSIONED, REBOOTED) use the same DeactivateNodeTransition class. The DeactivateNodeTransition class naturally decrements the active node, however the in cases where the node has transition to UNHEALTHY the active count has already been decremented.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1098">YARN-1098</a>.
|
|
|
+ Major sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>Separate out RM services into "Always On" and "Active"</b><br>
|
|
|
+ <blockquote>From discussion on YARN-1027, it makes sense to separate out services that are stateful and stateless. The stateless services can run perennially irrespective of whether the RM is in Active/Standby state, while the stateful services need to be started on transitionToActive() and completely shutdown on transitionToStandby().
|
|
|
+
|
|
|
+The external-facing stateless services should respond to the client/AM/NM requests depending on whether the RM is Active/Standby.
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1068">YARN-1068</a>.
|
|
|
+ Major sub-task reported by Karthik Kambatla and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>Add admin support for HA operations</b><br>
|
|
|
+ <blockquote>Support HA admin operations to facilitate transitioning the RM to Active and Standby states.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1060">YARN-1060</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Niranjan Singh (scheduler)<br>
|
|
|
+ <b>Two tests in TestFairScheduler are missing @Test annotation</b><br>
|
|
|
+ <blockquote>Amazingly, these tests appear to pass with the annotations added.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1053">YARN-1053</a>.
|
|
|
+ Blocker bug reported by Omkar Vinit Joshi and fixed by Omkar Vinit Joshi <br>
|
|
|
+ <b>Diagnostic message from ContainerExitEvent is ignored in ContainerImpl</b><br>
|
|
|
+ <blockquote>If the container launch fails then we send ContainerExitEvent. This event contains exitCode and diagnostic message. Today we are ignoring diagnostic message while handling this event inside ContainerImpl. Fixing it as it is useful in diagnosing the failure.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1044">YARN-1044</a>.
|
|
|
+ Critical bug reported by Sangjin Lee and fixed by Sangjin Lee (resourcemanager , scheduler)<br>
|
|
|
+ <b>used/min/max resources do not display info in the scheduler page</b><br>
|
|
|
+ <blockquote>Go to the scheduler page in RM, and click any queue to display the detailed info. You'll find that none of the resources entries (used, min, or max) would display values.
|
|
|
+
|
|
|
+It is because the values contain brackets ("<" and ">") and are not properly html-escaped.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1033">YARN-1033</a>.
|
|
|
+ Major sub-task reported by Nemon Lou and fixed by Karthik Kambatla <br>
|
|
|
+ <b>Expose RM active/standby state to Web UI and REST API</b><br>
|
|
|
+ <blockquote>Both active and standby RM shall expose it's web server and show it's current state (active or standby) on web page. Users should be able to access this information through the REST API as well.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1029">YARN-1029</a>.
|
|
|
+ Major sub-task reported by Bikas Saha and fixed by Karthik Kambatla <br>
|
|
|
+ <b>Allow embedding leader election into the RM</b><br>
|
|
|
+ <blockquote>It should be possible to embed common ActiveStandyElector into the RM such that ZooKeeper based leader election and notification is in-built. In conjunction with a ZK state store, this configuration will be a simple deployment option.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1028">YARN-1028</a>.
|
|
|
+ Major sub-task reported by Bikas Saha and fixed by Karthik Kambatla <br>
|
|
|
+ <b>Add FailoverProxyProvider like capability to RMProxy</b><br>
|
|
|
+ <blockquote>RMProxy layer currently abstracts RM discovery and implements it by looking up service information from configuration. Motivated by HDFS and using existing classes from Common, we can add failover proxy providers that may provide RM discovery in extensible ways.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1027">YARN-1027</a>.
|
|
|
+ Major sub-task reported by Bikas Saha and fixed by Karthik Kambatla <br>
|
|
|
+ <b>Implement RMHAProtocolService</b><br>
|
|
|
+ <blockquote>Implement existing HAServiceProtocol from Hadoop common. This protocol is the single point of interaction between the RM and HA clients/services.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1022">YARN-1022</a>.
|
|
|
+ Trivial bug reported by Bikas Saha and fixed by haosdent <br>
|
|
|
+ <b>Unnecessary INFO logs in AMRMClientAsync</b><br>
|
|
|
+ <blockquote>Logs like the following should be debug or else every legitimate stop causes unnecessary exception traces in the logs.
|
|
|
+
|
|
|
+464 2013-08-03 20:01:34,459 INFO [AMRM Heartbeater thread] org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl: Heartbeater interrupted
|
|
|
+465 java.lang.InterruptedException: sleep interrupted
|
|
|
+466 at java.lang.Thread.sleep(Native Method)
|
|
|
+467 at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$HeartbeatThread.run(AMRMClientAsyncImpl.java:249)
|
|
|
+468 2013-08-03 20:01:34,460 INFO [AMRM Callback Handler Thread] org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl: Interrupted while waiting for queue
|
|
|
+469 java.lang.InterruptedException
|
|
|
+470 at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer. java:1961)
|
|
|
+471 at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1996)
|
|
|
+472 at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
|
|
|
+473 at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:275)</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1021">YARN-1021</a>.
|
|
|
+ Major new feature reported by Wei Yan and fixed by Wei Yan (scheduler)<br>
|
|
|
+ <b>Yarn Scheduler Load Simulator</b><br>
|
|
|
+ <blockquote>The Yarn Scheduler is a fertile area of interest with different implementations, e.g., Fifo, Capacity and Fair schedulers. Meanwhile, several optimizations are also made to improve scheduler performance for different scenarios and workload. Each scheduler algorithm has its own set of features, and drives scheduling decisions by many factors, such as fairness, capacity guarantee, resource availability, etc. It is very important to evaluate a scheduler algorithm very well before we deploy it in a production cluster. Unfortunately, currently it is non-trivial to evaluate a scheduling algorithm. Evaluating in a real cluster is always time and cost consuming, and it is also very hard to find a large-enough cluster. Hence, a simulator which can predict how well a scheduler algorithm for some specific workload would be quite useful.
|
|
|
+
|
|
|
+We want to build a Scheduler Load Simulator to simulate large-scale Yarn clusters and application loads in a single machine. This would be invaluable in furthering Yarn by providing a tool for researchers and developers to prototype new scheduler features and predict their behavior and performance with reasonable amount of confidence, there-by aiding rapid innovation.
|
|
|
+
|
|
|
+The simulator will exercise the real Yarn ResourceManager removing the network factor by simulating NodeManagers and ApplicationMasters via handling and dispatching NM/AMs heartbeat events from within the same JVM.
|
|
|
+
|
|
|
+To keep tracking of scheduler behavior and performance, a scheduler wrapper will wrap the real scheduler.
|
|
|
+
|
|
|
+The simulator will produce real time metrics while executing, including:
|
|
|
+
|
|
|
+* Resource usages for whole cluster and each queue, which can be utilized to configure cluster and queue's capacity.
|
|
|
+* The detailed application execution trace (recorded in relation to simulated time), which can be analyzed to understand/validate the scheduler behavior (individual jobs turn around time, throughput, fairness, capacity guarantee, etc).
|
|
|
+* Several key metrics of scheduler algorithm, such as time cost of each scheduler operation (allocate, handle, etc), which can be utilized by Hadoop developers to find the code spots and scalability limits.
|
|
|
+
|
|
|
+The simulator will provide real time charts showing the behavior of the scheduler and its performance.
|
|
|
+
|
|
|
+A short demo is available http://www.youtube.com/watch?v=6thLi8q0qLE, showing how to use simulator to simulate Fair Scheduler and Capacity Scheduler.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-1010">YARN-1010</a>.
|
|
|
+ Critical improvement reported by Alejandro Abdelnur and fixed by Wei Yan (scheduler)<br>
|
|
|
+ <b>FairScheduler: decouple container scheduling from nodemanager heartbeats</b><br>
|
|
|
+ <blockquote>Currently scheduling for a node is done when a node heartbeats.
|
|
|
+
|
|
|
+For large cluster where the heartbeat interval is set to several seconds this delays scheduling of incoming allocations significantly.
|
|
|
+
|
|
|
+We could have a continuous loop scanning all nodes and doing scheduling. If there is availability AMs will get the allocation in the next heartbeat after the one that placed the request.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-985">YARN-985</a>.
|
|
|
+ Major improvement reported by Ravi Prakash and fixed by Ravi Prakash (nodemanager)<br>
|
|
|
+ <b>Nodemanager should log where a resource was localized</b><br>
|
|
|
+ <blockquote>When a resource is localized, we should log WHERE on the local disk it was localized. This helps in debugging afterwards (e.g. if the disk was to go bad).</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-976">YARN-976</a>.
|
|
|
+ Major sub-task reported by Sandy Ryza and fixed by Sandy Ryza (documentation)<br>
|
|
|
+ <b>Document the meaning of a virtual core</b><br>
|
|
|
+ <blockquote>As virtual cores are a somewhat novel concept, it would be helpful to have thorough documentation that clarifies their meaning.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-895">YARN-895</a>.
|
|
|
+ Major sub-task reported by Jian He and fixed by Jian He (resourcemanager)<br>
|
|
|
+ <b>RM crashes if it restarts while the state-store is down</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-891">YARN-891</a>.
|
|
|
+ Major sub-task reported by Bikas Saha and fixed by Jian He (resourcemanager)<br>
|
|
|
+ <b>Store completed application information in RM state store</b><br>
|
|
|
+ <blockquote>Store completed application/attempt info in RMStateStore when application/attempt completes. This solves some problems like finished application get lost after RM restart and some other races like YARN-1195</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-888">YARN-888</a>.
|
|
|
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur <br>
|
|
|
+ <b>clean up POM dependencies</b><br>
|
|
|
+ <blockquote>Intermediate 'pom' modules define dependencies inherited by leaf modules.
|
|
|
+
|
|
|
+This is causing issues in intellij IDE.
|
|
|
+
|
|
|
+We should normalize the leaf modules like in common, hdfs and tools where all dependencies are defined in each leaf module and the intermediate 'pom' module do not define any dependency.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-879">YARN-879</a>.
|
|
|
+ Major bug reported by Junping Du and fixed by Junping Du <br>
|
|
|
+ <b>Fix tests w.r.t o.a.h.y.server.resourcemanager.Application</b><br>
|
|
|
+ <blockquote>getResources() will return a list of containers that allocated by RM. However, it is now return null directly. The worse thing is: if LOG.debug is enabled, then it will definitely cause NPE exception.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-819">YARN-819</a>.
|
|
|
+ Major sub-task reported by Robert Parker and fixed by Robert Parker (nodemanager , resourcemanager)<br>
|
|
|
+ <b>ResourceManager and NodeManager should check for a minimum allowed version</b><br>
|
|
|
+ <blockquote>Our use case is during upgrade on a large cluster several NodeManagers may not restart with the new version. Once the RM comes back up the NodeManager will re-register without issue to the RM.
|
|
|
+
|
|
|
+The NM should report the version the RM. The RM should have a configuration to disallow the check (default), equal to the RM (to prevent config change for each release), equal to or greater than RM (to allow NM upgrades), and finally an explicit version or version range.
|
|
|
+
|
|
|
+The RM should also have an configuration on how to treat the mismatch: REJECT, or REBOOT the NM.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-807">YARN-807</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Sandy Ryza <br>
|
|
|
+ <b>When querying apps by queue, iterating over all apps is inefficient and limiting </b><br>
|
|
|
+ <blockquote>The question "which apps are in queue x" can be asked via the RM REST APIs, through the ClientRMService, and through the command line. In all these cases, the question is answered by scanning through every RMApp and filtering by the app's queue name.
|
|
|
+
|
|
|
+All schedulers maintain a mapping of queues to applications. I think it would make more sense to ask the schedulers which applications are in a given queue. This is what was done in MR1. This would also have the advantage of allowing a parent queue to return all the applications on leaf queues under it, and allow queue name aliases, as in the way that "root.default" and "default" refer to the same queue in the fair scheduler.
|
|
|
+
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-786">YARN-786</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Sandy Ryza <br>
|
|
|
+ <b>Expose application resource usage in RM REST API</b><br>
|
|
|
+ <blockquote>It might be good to require users to explicitly ask for this information, as it's a little more expensive to collect than the other fields in AppInfo.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-764">YARN-764</a>.
|
|
|
+ Major bug reported by Nemon Lou and fixed by Nemon Lou (resourcemanager)<br>
|
|
|
+ <b>blank Used Resources on Capacity Scheduler page </b><br>
|
|
|
+ <blockquote>Even when there are jobs running,used resources is empty on Capacity Scheduler page for leaf queue.(I use google-chrome on windows 7.)
|
|
|
+After changing resource.java's toString method by replacing "<>" with "{}",this bug gets fixed.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-709">YARN-709</a>.
|
|
|
+ Major sub-task reported by Jian He and fixed by Jian He (resourcemanager)<br>
|
|
|
+ <b>verify that new jobs submitted with old RM delegation tokens after RM restart are accepted</b><br>
|
|
|
+ <blockquote>More elaborate test for restoring RM delegation tokens on RM restart.
|
|
|
+New jobs with old RM delegation tokens should be accepted by new RM as long as the token is still valid</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-674">YARN-674</a>.
|
|
|
+ Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Omkar Vinit Joshi (resourcemanager)<br>
|
|
|
+ <b>Slow or failing DelegationToken renewals on submission itself make RM unavailable</b><br>
|
|
|
+ <blockquote>This was caused by YARN-280. A slow or a down NameNode for will make it look like RM is unavailable as it may run out of RPC handlers due to blocked client submissions.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-649">YARN-649</a>.
|
|
|
+ Major sub-task reported by Sandy Ryza and fixed by Sandy Ryza (nodemanager)<br>
|
|
|
+ <b>Make container logs available over HTTP in plain text</b><br>
|
|
|
+ <blockquote>It would be good to make container logs available over the REST API for MAPREDUCE-4362 and so that they can be accessed programatically in general.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-584">YARN-584</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Harshit Daga (scheduler)<br>
|
|
|
+ <b>In scheduler web UIs, queues unexpand on refresh</b><br>
|
|
|
+ <blockquote>In the fair scheduler web UI, you can expand queue information. Refreshing the page causes the expansions to go away, which is annoying for someone who wants to monitor the scheduler page and needs to reopen all the queues they care about each time.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-546">YARN-546</a>.
|
|
|
+ Major bug reported by Lohit Vijayarenu and fixed by Sandy Ryza (scheduler)<br>
|
|
|
+ <b>Allow disabling the Fair Scheduler event log</b><br>
|
|
|
+ <blockquote>Hadoop 1.0 supported an option to turn on/off FairScheduler event logging using mapred.fairscheduler.eventlog.enabled. In Hadoop 2.0, it looks like this option has been removed (or not ported?) which causes event logging to be enabled by default and there is no way to turn it off.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-478">YARN-478</a>.
|
|
|
+ Major sub-task reported by Aleksey Gorshkov and fixed by Aleksey Gorshkov <br>
|
|
|
+ <b>fix coverage org.apache.hadoop.yarn.webapp.log</b><br>
|
|
|
+ <blockquote>fix coverage org.apache.hadoop.yarn.webapp.log
|
|
|
+one patch for trunk, branch-2, branch-0.23</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-465">YARN-465</a>.
|
|
|
+ Major sub-task reported by Aleksey Gorshkov and fixed by Andrey Klochkov <br>
|
|
|
+ <b>fix coverage org.apache.hadoop.yarn.server.webproxy</b><br>
|
|
|
+ <blockquote>fix coverage org.apache.hadoop.yarn.server.webproxy
|
|
|
+patch YARN-465-trunk.patch for trunk
|
|
|
+patch YARN-465-branch-2.patch for branch-2
|
|
|
+patch YARN-465-branch-0.23.patch for branch-0.23
|
|
|
+
|
|
|
+There is issue in branch-0.23 . Patch does not creating .keep file.
|
|
|
+For fix it need to run commands:
|
|
|
+
|
|
|
+mkdir yhadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/proxy
|
|
|
+touch yhadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/proxy/.keep
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-461">YARN-461</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Wei Yan (resourcemanager)<br>
|
|
|
+ <b>Fair scheduler should not accept apps with empty string queue name</b><br>
|
|
|
+ <blockquote>When an app is submitted with "" for the queue, the RMAppManager passes it on like it does with any other string.
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-427">YARN-427</a>.
|
|
|
+ Major sub-task reported by Aleksey Gorshkov and fixed by Aleksey Gorshkov <br>
|
|
|
+ <b>Coverage fix for org.apache.hadoop.yarn.server.api.*</b><br>
|
|
|
+ <blockquote>Coverage fix for org.apache.hadoop.yarn.server.api.*
|
|
|
+
|
|
|
+patch YARN-427-trunk.patch for trunk
|
|
|
+patch YARN-427-branch-2.patch for branch-2 and branch-0.23</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-425">YARN-425</a>.
|
|
|
+ Major sub-task reported by Aleksey Gorshkov and fixed by Aleksey Gorshkov <br>
|
|
|
+ <b>coverage fix for yarn api</b><br>
|
|
|
+ <blockquote>coverage fix for yarn api
|
|
|
+patch YARN-425-trunk-a.patch for trunk
|
|
|
+patch YARN-425-branch-2.patch for branch-2
|
|
|
+patch YARN-425-branch-0.23.patch for branch-0.23</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-408">YARN-408</a>.
|
|
|
+ Minor bug reported by Mayank Bansal and fixed by Mayank Bansal (scheduler)<br>
|
|
|
+ <b>Capacity Scheduler delay scheduling should not be disabled by default</b><br>
|
|
|
+ <blockquote>Capacity Scheduler delay scheduling should not be disabled by default.
|
|
|
+Enabling it to number of nodes in one rack.
|
|
|
+
|
|
|
+Thanks,
|
|
|
+Mayank</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-353">YARN-353</a>.
|
|
|
+ Major sub-task reported by Hitesh Shah and fixed by Karthik Kambatla (resourcemanager)<br>
|
|
|
+ <b>Add Zookeeper-based store implementation for RMStateStore</b><br>
|
|
|
+ <blockquote>Add store that write RM state data to ZK
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-312">YARN-312</a>.
|
|
|
+ Major sub-task reported by Junping Du and fixed by Junping Du (api)<br>
|
|
|
+ <b>Add updateNodeResource in ResourceManagerAdministrationProtocol</b><br>
|
|
|
+ <blockquote>Add fundamental RPC (ResourceManagerAdministrationProtocol) to support node's resource change. For design detail, please refer parent JIRA: YARN-291.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-311">YARN-311</a>.
|
|
|
+ Major sub-task reported by Junping Du and fixed by Junping Du (resourcemanager , scheduler)<br>
|
|
|
+ <b>Dynamic node resource configuration: core scheduler changes</b><br>
|
|
|
+ <blockquote>As the first step, we go for resource change on RM side and expose admin APIs (admin protocol, CLI, REST and JMX API) later. In this jira, we will only contain changes in scheduler.
|
|
|
+The flow to update node's resource and awareness in resource scheduling is:
|
|
|
+1. Resource update is through admin API to RM and take effect on RMNodeImpl.
|
|
|
+2. When next NM heartbeat for updating status comes, the RMNode's resource change will be aware and the delta resource is added to schedulerNode's availableResource before actual scheduling happens.
|
|
|
+3. Scheduler do resource allocation according to new availableResource in SchedulerNode.
|
|
|
+For more design details, please refer proposal and discussions in parent JIRA: YARN-291.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-305">YARN-305</a>.
|
|
|
+ Critical bug reported by Lohit Vijayarenu and fixed by Lohit Vijayarenu (resourcemanager)<br>
|
|
|
+ <b>Fair scheduler logs too many "Node offered to app:..." messages</b><br>
|
|
|
+ <blockquote>Running fair scheduler YARN shows that RM has lots of messages like the below.
|
|
|
+{noformat}
|
|
|
+INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AppSchedulable: Node offered to app: application_1357147147433_0002 reserved: false
|
|
|
+{noformat}
|
|
|
+
|
|
|
+They dont seem to tell much and same line is dumped many times in RM log. It would be good to have it improved with node information or moved to some other logging level with enough debug information</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/YARN-7">YARN-7</a>.
|
|
|
+ Major sub-task reported by Arun C Murthy and fixed by Junping Du <br>
|
|
|
+ <b>Add support for DistributedShell to ask for CPUs along with memory</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5744">MAPREDUCE-5744</a>.
|
|
|
+ Blocker bug reported by Sangjin Lee and fixed by Gera Shegalov <br>
|
|
|
+ <b>Job hangs because RMContainerAllocator$AssignedRequests.preemptReduce() violates the comparator contract</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5743">MAPREDUCE-5743</a>.
|
|
|
+ Major bug reported by Ted Yu and fixed by Ted Yu <br>
|
|
|
+ <b>TestRMContainerAllocator is failing</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5729">MAPREDUCE-5729</a>.
|
|
|
+ Critical bug reported by Karthik Kambatla and fixed by Karthik Kambatla (mrv2)<br>
|
|
|
+ <b>mapred job -list throws NPE</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5725">MAPREDUCE-5725</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Sandy Ryza <br>
|
|
|
+ <b>TestNetworkedJob relies on the Capacity Scheduler</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5724">MAPREDUCE-5724</a>.
|
|
|
+ Critical bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (jobhistoryserver)<br>
|
|
|
+ <b>JobHistoryServer does not start if HDFS is not running</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5723">MAPREDUCE-5723</a>.
|
|
|
+ Blocker bug reported by Mohammad Kamrul Islam and fixed by Mohammad Kamrul Islam (applicationmaster)<br>
|
|
|
+ <b>MR AM container log can be truncated or empty</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5694">MAPREDUCE-5694</a>.
|
|
|
+ Major bug reported by Mohammad Kamrul Islam and fixed by Mohammad Kamrul Islam <br>
|
|
|
+ <b>MR AM container syslog is empty </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5693">MAPREDUCE-5693</a>.
|
|
|
+ Major bug reported by Gera Shegalov and fixed by Gera Shegalov (mrv2)<br>
|
|
|
+ <b>Restore MRv1 behavior for log flush</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5692">MAPREDUCE-5692</a>.
|
|
|
+ Major improvement reported by Gera Shegalov and fixed by Gera Shegalov (mrv2)<br>
|
|
|
+ <b>Add explicit diagnostics when a task attempt is killed due to speculative execution</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5689">MAPREDUCE-5689</a>.
|
|
|
+ Critical bug reported by Lohit Vijayarenu and fixed by Lohit Vijayarenu <br>
|
|
|
+ <b>MRAppMaster does not preempt reducers when scheduled maps cannot be fulfilled</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5687">MAPREDUCE-5687</a>.
|
|
|
+ Major test reported by Ted Yu and fixed by Jian He <br>
|
|
|
+ <b>TestYARNRunner#testResourceMgrDelegate fails with NPE after YARN-1446</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5685">MAPREDUCE-5685</a>.
|
|
|
+ Blocker bug reported by Yi Song and fixed by Yi Song (client)<br>
|
|
|
+ <b>getCacheFiles() api doesn't work in WrappedReducer.java due to typo</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5679">MAPREDUCE-5679</a>.
|
|
|
+ Major bug reported by Liyin Liang and fixed by Liyin Liang <br>
|
|
|
+ <b>TestJobHistoryParsing has race condition</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5674">MAPREDUCE-5674</a>.
|
|
|
+ Major bug reported by Chuan Liu and fixed by Chuan Liu (client)<br>
|
|
|
+ <b>Missing start and finish time in mapred.JobStatus</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5672">MAPREDUCE-5672</a>.
|
|
|
+ Major improvement reported by Gera Shegalov and fixed by Gera Shegalov (mr-am , mrv2)<br>
|
|
|
+ <b>Provide optional RollingFileAppender for container log4j (syslog)</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5656">MAPREDUCE-5656</a>.
|
|
|
+ Critical bug reported by Jason Lowe and fixed by Jason Lowe <br>
|
|
|
+ <b>bzip2 codec can drop records when reading data in splits</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5650">MAPREDUCE-5650</a>.
|
|
|
+ Major bug reported by Gera Shegalov and fixed by Gera Shegalov (mrv2)<br>
|
|
|
+ <b>Job fails when hprof mapreduce.task.profile.map/reduce.params is specified</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5645">MAPREDUCE-5645</a>.
|
|
|
+ Major bug reported by Jonathan Eagles and fixed by Mit Desai <br>
|
|
|
+ <b>TestFixedLengthInputFormat fails with native libs</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5640">MAPREDUCE-5640</a>.
|
|
|
+ Trivial improvement reported by Jason Lowe and fixed by Jason Lowe (test)<br>
|
|
|
+ <b>Rename TestLineRecordReader in jobclient module</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5632">MAPREDUCE-5632</a>.
|
|
|
+ Major test reported by Ted Yu and fixed by Jonathan Eagles <br>
|
|
|
+ <b>TestRMContainerAllocator#testUpdatedNodes fails</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5631">MAPREDUCE-5631</a>.
|
|
|
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles <br>
|
|
|
+ <b>TestJobEndNotifier.testNotifyRetries fails with Should have taken more than 5 seconds in jdk7</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5625">MAPREDUCE-5625</a>.
|
|
|
+ Major test reported by Jonathan Eagles and fixed by Mariappan Asokan <br>
|
|
|
+ <b>TestFixedLengthInputFormat fails in jdk7 environment</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5623">MAPREDUCE-5623</a>.
|
|
|
+ Major bug reported by Tsuyoshi OZAWA and fixed by Jason Lowe <br>
|
|
|
+ <b>TestJobCleanup fails because of RejectedExecutionException and NPE.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5616">MAPREDUCE-5616</a>.
|
|
|
+ Major bug reported by Chris Nauroth and fixed by Chris Nauroth (client)<br>
|
|
|
+ <b>MR Client-AppMaster RPC max retries on socket timeout is too high.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5613">MAPREDUCE-5613</a>.
|
|
|
+ Major bug reported by Gera Shegalov and fixed by Gera Shegalov (applicationmaster)<br>
|
|
|
+ <b>DefaultSpeculator holds and checks hashmap that is always empty</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5610">MAPREDUCE-5610</a>.
|
|
|
+ Major test reported by Jonathan Eagles and fixed by Jonathan Eagles <br>
|
|
|
+ <b>TestSleepJob fails in jdk7</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5604">MAPREDUCE-5604</a>.
|
|
|
+ Minor bug reported by Chris Nauroth and fixed by Chris Nauroth (test)<br>
|
|
|
+ <b>TestMRAMWithNonNormalizedCapabilities fails on Windows due to exceeding max path length</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5601">MAPREDUCE-5601</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Sandy Ryza <br>
|
|
|
+ <b>ShuffleHandler fadvises file regions as DONTNEED even when fetch fails</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5598">MAPREDUCE-5598</a>.
|
|
|
+ Major bug reported by Robert Kanter and fixed by Robert Kanter (test)<br>
|
|
|
+ <b>TestUserDefinedCounters.testMapReduceJob is flakey</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5596">MAPREDUCE-5596</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Sandy Ryza <br>
|
|
|
+ <b>Allow configuring the number of threads used to serve shuffle connections</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5587">MAPREDUCE-5587</a>.
|
|
|
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles <br>
|
|
|
+ <b>TestTextOutputFormat fails on JDK7</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5586">MAPREDUCE-5586</a>.
|
|
|
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles <br>
|
|
|
+ <b>TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5585">MAPREDUCE-5585</a>.
|
|
|
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles <br>
|
|
|
+ <b>TestCopyCommitter#testNoCommitAction Fails on JDK7</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5569">MAPREDUCE-5569</a>.
|
|
|
+ Major bug reported by Nathan Roberts and fixed by Nathan Roberts <br>
|
|
|
+ <b>FloatSplitter is not generating correct splits</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5561">MAPREDUCE-5561</a>.
|
|
|
+ Critical bug reported by Cindy Li and fixed by Karthik Kambatla <br>
|
|
|
+ <b>org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5550">MAPREDUCE-5550</a>.
|
|
|
+ Major bug reported by Vrushali C and fixed by Gera Shegalov <br>
|
|
|
+ <b>Task Status message (reporter.setStatus) not shown in UI with Hadoop 2.0</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5546">MAPREDUCE-5546</a>.
|
|
|
+ Major bug reported by Chuan Liu and fixed by Chuan Liu <br>
|
|
|
+ <b>mapred.cmd on Windows set HADOOP_OPTS incorrectly</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5522">MAPREDUCE-5522</a>.
|
|
|
+ Minor bug reported by Jinghui Wang and fixed by Jinghui Wang (test)<br>
|
|
|
+ <b>Incorrectly expect the array of JobQueueInfo returned by o.a.h.mapred.QueueManager#getJobQueueInfos to have a specific order.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5518">MAPREDUCE-5518</a>.
|
|
|
+ Trivial bug reported by Albert Chu and fixed by Albert Chu (examples)<br>
|
|
|
+ <b>Fix typo "can't read paritions file"</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5514">MAPREDUCE-5514</a>.
|
|
|
+ Blocker bug reported by Zhijie Shen and fixed by Zhijie Shen <br>
|
|
|
+ <b>TestRMContainerAllocator fails on trunk</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5504">MAPREDUCE-5504</a>.
|
|
|
+ Major bug reported by Thomas Graves and fixed by Kousuke Saruta (client)<br>
|
|
|
+ <b>mapred queue -info inconsistent with types</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5487">MAPREDUCE-5487</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Sandy Ryza (performance , task)<br>
|
|
|
+ <b>In task processes, JobConf is unnecessarily loaded again in Limits</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5484">MAPREDUCE-5484</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Sandy Ryza (task)<br>
|
|
|
+ <b>YarnChild unnecessarily loads job conf twice</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5481">MAPREDUCE-5481</a>.
|
|
|
+ Blocker bug reported by Jason Lowe and fixed by Sandy Ryza (mrv2 , test)<br>
|
|
|
+ <b>Enable uber jobs to have multiple reducers </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5464">MAPREDUCE-5464</a>.
|
|
|
+ Major task reported by Sandy Ryza and fixed by Sandy Ryza <br>
|
|
|
+ <b>Add analogs of the SLOTS_MILLIS counters that jive with the YARN resource model</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5463">MAPREDUCE-5463</a>.
|
|
|
+ Major task reported by Sandy Ryza and fixed by Tsuyoshi OZAWA <br>
|
|
|
+ <b>Deprecate SLOTS_MILLIS counters</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5457">MAPREDUCE-5457</a>.
|
|
|
+ Major improvement reported by Sandy Ryza and fixed by Sandy Ryza <br>
|
|
|
+ <b>Add a KeyOnlyTextOutputReader to enable streaming to write out text files without separators</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5451">MAPREDUCE-5451</a>.
|
|
|
+ Major bug reported by Mostafa Elhemali and fixed by Yingda Chen <br>
|
|
|
+ <b>MR uses LD_LIBRARY_PATH which doesn't mean anything in Windows</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5431">MAPREDUCE-5431</a>.
|
|
|
+ Major bug reported by Timothy St. Clair and fixed by Timothy St. Clair (build)<br>
|
|
|
+ <b>Missing pom dependency in MR-client</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5411">MAPREDUCE-5411</a>.
|
|
|
+ Major sub-task reported by Ashwin Shankar and fixed by Ashwin Shankar (jobhistoryserver)<br>
|
|
|
+ <b>Refresh size of loaded job cache on history server</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5409">MAPREDUCE-5409</a>.
|
|
|
+ Major sub-task reported by Devaraj K and fixed by Gera Shegalov <br>
|
|
|
+ <b>MRAppMaster throws InvalidStateTransitonException: Invalid event: TA_TOO_MANY_FETCH_FAILURE at KILLED for TaskAttemptImpl</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5404">MAPREDUCE-5404</a>.
|
|
|
+ Major bug reported by Ted Yu and fixed by Ted Yu (jobhistoryserver)<br>
|
|
|
+ <b>HSAdminServer does not use ephemeral ports in minicluster mode</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5386">MAPREDUCE-5386</a>.
|
|
|
+ Major sub-task reported by Ashwin Shankar and fixed by Ashwin Shankar (jobhistoryserver)<br>
|
|
|
+ <b>Ability to refresh history server job retention and job cleaner settings</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5380">MAPREDUCE-5380</a>.
|
|
|
+ Major bug reported by Stephen Chu and fixed by Stephen Chu <br>
|
|
|
+ <b>Invalid mapred command should return non-zero exit code</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5373">MAPREDUCE-5373</a>.
|
|
|
+ Major bug reported by Chuan Liu and fixed by Jonathan Eagles <br>
|
|
|
+ <b>TestFetchFailure.testFetchFailureMultipleReduces could fail intermittently</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5356">MAPREDUCE-5356</a>.
|
|
|
+ Major sub-task reported by Ashwin Shankar and fixed by Ashwin Shankar (jobhistoryserver)<br>
|
|
|
+ <b>Ability to refresh aggregated log retention period and check interval </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5332">MAPREDUCE-5332</a>.
|
|
|
+ Major new feature reported by Jason Lowe and fixed by Jason Lowe (jobhistoryserver)<br>
|
|
|
+ <b>Support token-preserving restart of history server</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5329">MAPREDUCE-5329</a>.
|
|
|
+ Major bug reported by Avner BenHanoch and fixed by Avner BenHanoch (mr-am)<br>
|
|
|
+ <b>APPLICATION_INIT is never sent to AuxServices other than the builtin ShuffleHandler</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5316">MAPREDUCE-5316</a>.
|
|
|
+ Major bug reported by Ashwin Shankar and fixed by Ashwin Shankar (client)<br>
|
|
|
+ <b>job -list-attempt-ids command does not handle illegal task-state</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5266">MAPREDUCE-5266</a>.
|
|
|
+ Major new feature reported by Jason Lowe and fixed by Ashwin Shankar (jobhistoryserver)<br>
|
|
|
+ <b>Ability to refresh retention settings on history server</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5265">MAPREDUCE-5265</a>.
|
|
|
+ Major new feature reported by Jason Lowe and fixed by Ashwin Shankar (jobhistoryserver)<br>
|
|
|
+ <b>History server admin service to refresh user and superuser group mappings</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5186">MAPREDUCE-5186</a>.
|
|
|
+ Critical bug reported by Sangjin Lee and fixed by Robert Parker (job submission)<br>
|
|
|
+ <b>mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5102">MAPREDUCE-5102</a>.
|
|
|
+ Major test reported by Aleksey Gorshkov and fixed by Andrey Klochkov <br>
|
|
|
+ <b>fix coverage org.apache.hadoop.mapreduce.lib.db and org.apache.hadoop.mapred.lib.db</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5084">MAPREDUCE-5084</a>.
|
|
|
+ Major test reported by Aleksey Gorshkov and fixed by Aleksey Gorshkov <br>
|
|
|
+ <b>fix coverage org.apache.hadoop.mapreduce.v2.app.webapp and org.apache.hadoop.mapreduce.v2.hs.webapp</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5052">MAPREDUCE-5052</a>.
|
|
|
+ Critical bug reported by Kendall Thrapp and fixed by Chen He (jobhistoryserver , webapps)<br>
|
|
|
+ <b>Job History UI and web services confusing job start time and job submit time</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5020">MAPREDUCE-5020</a>.
|
|
|
+ Major bug reported by Trevor Robinson and fixed by Trevor Robinson (client)<br>
|
|
|
+ <b>Compile failure with JDK8</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4680">MAPREDUCE-4680</a>.
|
|
|
+ Major bug reported by Sandy Ryza and fixed by Robert Kanter (jobhistoryserver)<br>
|
|
|
+ <b>Job history cleaner should only check timestamps of files in old enough directories</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4421">MAPREDUCE-4421</a>.
|
|
|
+ Major improvement reported by Arun C Murthy and fixed by Jason Lowe <br>
|
|
|
+ <b>Run MapReduce framework via the distributed cache</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3310">MAPREDUCE-3310</a>.
|
|
|
+ Major improvement reported by Mathias Herberts and fixed by Alejandro Abdelnur (client)<br>
|
|
|
+ <b>Custom grouping comparator cannot be set for Combiners</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1176">MAPREDUCE-1176</a>.
|
|
|
+ Major new feature reported by BitsOfInfo and fixed by Mariappan Asokan <br>
|
|
|
+ <b>FixedLengthInputFormat and FixedLengthRecordReader</b><br>
|
|
|
+ <blockquote>Addition of FixedLengthInputFormat and FixedLengthRecordReader in the org.apache.hadoop.mapreduce.lib.input package. These two classes can be used when you need to read data from files containing fixed length (fixed width) records. Such files have no CR/LF (or any combination thereof), no delimiters etc, but each record is a fixed length, and extra data is padded with spaces. The data is one gigantic line within a file. When creating a job that specifies this input format, the job must have the "mapreduce.input.fixedlengthinputformat.record.length" property set as follows myJobConf.setInt("mapreduce.input.fixedlengthinputformat.record.length",[myFixedRecordLength]);
|
|
|
+
|
|
|
+Please see javadoc for more details.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-434">MAPREDUCE-434</a>.
|
|
|
+ Minor improvement reported by Yoram Arnon and fixed by Aaron Kimball <br>
|
|
|
+ <b>LocalJobRunner limited to single reducer</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5921">HDFS-5921</a>.
|
|
|
+ Critical bug reported by Aaron T. Myers and fixed by Aaron T. Myers (namenode)<br>
|
|
|
+ <b>Cannot browse file system via NN web UI if any directory has the sticky bit set</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5876">HDFS-5876</a>.
|
|
|
+ Major bug reported by Haohui Mai and fixed by Haohui Mai (datanode)<br>
|
|
|
+ <b>SecureDataNodeStarter does not pick up configuration in hdfs-site.xml</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5873">HDFS-5873</a>.
|
|
|
+ Major bug reported by Yesha Vora and fixed by Haohui Mai <br>
|
|
|
+ <b>dfs.http.policy should have higher precedence over dfs.https.enable</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5845">HDFS-5845</a>.
|
|
|
+ Blocker bug reported by Andrew Wang and fixed by Andrew Wang (namenode)<br>
|
|
|
+ <b>SecondaryNameNode dies when checkpointing with cache pools</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5844">HDFS-5844</a>.
|
|
|
+ Minor bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)<br>
|
|
|
+ <b>Fix broken link in WebHDFS.apt.vm</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5842">HDFS-5842</a>.
|
|
|
+ Major bug reported by Arpit Gupta and fixed by Jing Zhao (security)<br>
|
|
|
+ <b>Cannot create hftp filesystem when using a proxy user ugi and a doAs on a secure cluster</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5841">HDFS-5841</a>.
|
|
|
+ Major improvement reported by Andrew Wang and fixed by Andrew Wang <br>
|
|
|
+ <b>Update HDFS caching documentation with new changes</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5837">HDFS-5837</a>.
|
|
|
+ Major bug reported by Bryan Beaudreault and fixed by Tao Luo (namenode)<br>
|
|
|
+ <b>dfs.namenode.replication.considerLoad does not consider decommissioned nodes</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5833">HDFS-5833</a>.
|
|
|
+ Trivial improvement reported by Bangtao Zhou and fixed by (namenode)<br>
|
|
|
+ <b>SecondaryNameNode have an incorrect java doc</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5830">HDFS-5830</a>.
|
|
|
+ Blocker bug reported by Yongjun Zhang and fixed by Yongjun Zhang (caching , hdfs-client)<br>
|
|
|
+ <b>WebHdfsFileSystem.getFileBlockLocations throws IllegalArgumentException when accessing another cluster. </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5825">HDFS-5825</a>.
|
|
|
+ Minor improvement reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Use FileUtils.copyFile() to implement DFSTestUtils.copyFile()</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5806">HDFS-5806</a>.
|
|
|
+ Major bug reported by Nathan Roberts and fixed by Nathan Roberts (balancer)<br>
|
|
|
+ <b>balancer should set SoTimeout to avoid indefinite hangs</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5800">HDFS-5800</a>.
|
|
|
+ Trivial bug reported by Kousuke Saruta and fixed by Kousuke Saruta (hdfs-client)<br>
|
|
|
+ <b>Typo: soft-limit for hard-limit in DFSClient</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5789">HDFS-5789</a>.
|
|
|
+ Major bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (namenode)<br>
|
|
|
+ <b>Some of snapshot APIs missing checkOperation double check in fsn</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5788">HDFS-5788</a>.
|
|
|
+ Major improvement reported by Nathan Roberts and fixed by Nathan Roberts (namenode)<br>
|
|
|
+ <b>listLocatedStatus response can be very large</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5784">HDFS-5784</a>.
|
|
|
+ Major sub-task reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (namenode)<br>
|
|
|
+ <b>reserve space in edit log header and fsimage header for feature flag section</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5777">HDFS-5777</a>.
|
|
|
+ Major bug reported by Jing Zhao and fixed by Jing Zhao (namenode)<br>
|
|
|
+ <b>Update LayoutVersion for the new editlog op OP_ADD_BLOCK</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5766">HDFS-5766</a>.
|
|
|
+ Major bug reported by Liang Xie and fixed by Liang Xie (hdfs-client)<br>
|
|
|
+ <b>In DFSInputStream, do not add datanode to deadNodes after InvalidEncryptionKeyException in fetchBlockByteRange</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5762">HDFS-5762</a>.
|
|
|
+ Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe <br>
|
|
|
+ <b>BlockReaderLocal doesn't return -1 on EOF when doing zero-length reads</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5756">HDFS-5756</a>.
|
|
|
+ Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (libhdfs)<br>
|
|
|
+ <b>hadoopRzOptionsSetByteBufferPool does not accept NULL argument, contrary to docs</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5748">HDFS-5748</a>.
|
|
|
+ Major improvement reported by Kihwal Lee and fixed by Haohui Mai <br>
|
|
|
+ <b>Too much information shown in the dfs health page.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5747">HDFS-5747</a>.
|
|
|
+ Minor bug reported by Tsz Wo (Nicholas), SZE and fixed by Arpit Agarwal (namenode)<br>
|
|
|
+ <b>BlocksMap.getStoredBlock(..) and BlockInfoUnderConstruction.addReplicaIfNotPresent(..) may throw NullPointerException</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5728">HDFS-5728</a>.
|
|
|
+ Critical bug reported by Vinayakumar B and fixed by Vinayakumar B (datanode)<br>
|
|
|
+ <b>[Diskfull] Block recovery will fail if the metafile does not have crc for all chunks of the block</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5721">HDFS-5721</a>.
|
|
|
+ Minor improvement reported by Ted Yu and fixed by Ted Yu <br>
|
|
|
+ <b>sharedEditsImage in Namenode#initializeSharedEdits() should be closed before method returns</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5719">HDFS-5719</a>.
|
|
|
+ Minor bug reported by Ted Yu and fixed by Ted Yu (namenode)<br>
|
|
|
+ <b>FSImage#doRollback() should close prevState before return</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5710">HDFS-5710</a>.
|
|
|
+ Major bug reported by Ted Yu and fixed by Uma Maheswara Rao G <br>
|
|
|
+ <b>FSDirectory#getFullPathName should check inodes against null</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5704">HDFS-5704</a>.
|
|
|
+ Major bug reported by Suresh Srinivas and fixed by Jing Zhao (namenode)<br>
|
|
|
+ <b>Change OP_UPDATE_BLOCKS with a new OP_ADD_BLOCK</b><br>
|
|
|
+ <blockquote>Add a new editlog record (OP_ADD_BLOCK) that only records allocation of the new block instead of the entire block list, on every block allocation.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5703">HDFS-5703</a>.
|
|
|
+ Major new feature reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (webhdfs)<br>
|
|
|
+ <b>Add support for HTTPS and swebhdfs to HttpFS</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5695">HDFS-5695</a>.
|
|
|
+ Major improvement reported by Haohui Mai and fixed by Haohui Mai (test)<br>
|
|
|
+ <b>Clean up TestOfflineEditsViewer and OfflineEditsViewerHelper</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5691">HDFS-5691</a>.
|
|
|
+ Minor bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)<br>
|
|
|
+ <b>Fix typo in ShortCircuitLocalRead document</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5690">HDFS-5690</a>.
|
|
|
+ Blocker bug reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>DataNode fails to start in secure mode when dfs.http.policy equals to HTTP_ONLY</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5681">HDFS-5681</a>.
|
|
|
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (namenode)<br>
|
|
|
+ <b>renewLease should not hold fsn write lock</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5677">HDFS-5677</a>.
|
|
|
+ Minor improvement reported by Vincent Sheffer and fixed by Vincent Sheffer (datanode , ha)<br>
|
|
|
+ <b>Need error checking for HA cluster configuration</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5676">HDFS-5676</a>.
|
|
|
+ Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (hdfs-client)<br>
|
|
|
+ <b>fix inconsistent synchronization of CachingStrategy</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5675">HDFS-5675</a>.
|
|
|
+ Minor bug reported by Plamen Jeliazkov and fixed by Plamen Jeliazkov (benchmarks)<br>
|
|
|
+ <b>Add Mkdirs operation to NNThroughputBenchmark</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5674">HDFS-5674</a>.
|
|
|
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (namenode)<br>
|
|
|
+ <b>Editlog code cleanup</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5671">HDFS-5671</a>.
|
|
|
+ Critical bug reported by JamesLi and fixed by JamesLi (hdfs-client)<br>
|
|
|
+ <b>Fix socket leak in DFSInputStream#getBlockReader</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5667">HDFS-5667</a>.
|
|
|
+ Major sub-task reported by Eric Sirianni and fixed by Arpit Agarwal (datanode)<br>
|
|
|
+ <b>Include DatanodeStorage in StorageReport</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5666">HDFS-5666</a>.
|
|
|
+ Minor bug reported by Colin Patrick McCabe and fixed by Jimmy Xiang (namenode)<br>
|
|
|
+ <b>Fix inconsistent synchronization in BPOfferService</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5663">HDFS-5663</a>.
|
|
|
+ Major improvement reported by Liang Xie and fixed by Liang Xie (hdfs-client)<br>
|
|
|
+ <b>make the retry time and interval value configurable in openInfo()</b><br>
|
|
|
+ <blockquote>Makes the retries and time between retries getting the length of the last block on file configurable. Below are the new configurations.
|
|
|
+
|
|
|
+dfs.client.retry.times.get-last-block-length
|
|
|
+dfs.client.retry.interval-ms.get-last-block-length
|
|
|
+
|
|
|
+They are set to the 3 and 4000 respectively, these being what was previously hardcoded.
|
|
|
+
|
|
|
+</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5662">HDFS-5662</a>.
|
|
|
+ Major improvement reported by Brandon Li and fixed by Brandon Li (namenode)<br>
|
|
|
+ <b>Can't decommission a DataNode due to file's replication factor larger than the rest of the cluster size</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5661">HDFS-5661</a>.
|
|
|
+ Major bug reported by Benoy Antony and fixed by Benoy Antony <br>
|
|
|
+ <b>Browsing FileSystem via web ui, should use datanode's fqdn instead of ip address</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5657">HDFS-5657</a>.
|
|
|
+ Major bug reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>race condition causes writeback state error in NFS gateway</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5652">HDFS-5652</a>.
|
|
|
+ Minor improvement reported by Liang Xie and fixed by Liang Xie (hdfs-client)<br>
|
|
|
+ <b>refactoring/uniforming invalid block token exception handling in DFSInputStream</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5649">HDFS-5649</a>.
|
|
|
+ Major bug reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>Unregister NFS and Mount service when NFS gateway is shutting down</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5637">HDFS-5637</a>.
|
|
|
+ Major improvement reported by Liang Xie and fixed by Liang Xie (hdfs-client , security)<br>
|
|
|
+ <b>try to refeatchToken while local read InvalidToken occurred</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5634">HDFS-5634</a>.
|
|
|
+ Major sub-task reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (hdfs-client)<br>
|
|
|
+ <b>allow BlockReaderLocal to switch between checksumming and not</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5633">HDFS-5633</a>.
|
|
|
+ Minor improvement reported by Jing Zhao and fixed by Jing Zhao <br>
|
|
|
+ <b>Improve OfflineImageViewer to use less memory</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5629">HDFS-5629</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Support HTTPS in JournalNode and SecondaryNameNode</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5592">HDFS-5592</a>.
|
|
|
+ Major bug reported by Vinayakumar B and fixed by Vinayakumar B <br>
|
|
|
+ <b>"DIR* completeFile: /file is closed by DFSClient_" should be logged only for successful closure of the file.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5590">HDFS-5590</a>.
|
|
|
+ Major bug reported by Jing Zhao and fixed by Jing Zhao <br>
|
|
|
+ <b>Block ID and generation stamp may be reused when persistBlocks is set to false</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5587">HDFS-5587</a>.
|
|
|
+ Minor improvement reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>add debug information when NFS fails to start with duplicate user or group names</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5582">HDFS-5582</a>.
|
|
|
+ Minor bug reported by Henry Hung and fixed by sathish <br>
|
|
|
+ <b>hdfs getconf -excludeFile or -includeFile always failed</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5581">HDFS-5581</a>.
|
|
|
+ Major bug reported by Vinayakumar B and fixed by Vinayakumar B (namenode)<br>
|
|
|
+ <b>NameNodeFsck should use only one instance of BlockPlacementPolicy</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5580">HDFS-5580</a>.
|
|
|
+ Major bug reported by Binglin Chang and fixed by Binglin Chang <br>
|
|
|
+ <b>Infinite loop in Balancer.waitForMoveCompletion</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5579">HDFS-5579</a>.
|
|
|
+ Major bug reported by zhaoyunjiong and fixed by zhaoyunjiong (namenode)<br>
|
|
|
+ <b>Under construction files make DataNode decommission take very long hours</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5577">HDFS-5577</a>.
|
|
|
+ Trivial improvement reported by Brandon Li and fixed by Brandon Li (documentation)<br>
|
|
|
+ <b>NFS user guide update</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5568">HDFS-5568</a>.
|
|
|
+ Major improvement reported by Vinayakumar B and fixed by Vinayakumar B (snapshots)<br>
|
|
|
+ <b>Support inclusion of snapshot paths in Namenode fsck</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5563">HDFS-5563</a>.
|
|
|
+ Major improvement reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>NFS gateway should commit the buffered data when read request comes after write to the same file</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5561">HDFS-5561</a>.
|
|
|
+ Minor improvement reported by Fengdong Yu and fixed by Haohui Mai (namenode)<br>
|
|
|
+ <b>FSNameSystem#getNameJournalStatus() in JMX should return plain text instead of HTML</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5560">HDFS-5560</a>.
|
|
|
+ Major bug reported by Josh Elser and fixed by Josh Elser <br>
|
|
|
+ <b>Trash configuration log statements prints incorrect units</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5558">HDFS-5558</a>.
|
|
|
+ Major bug reported by Kihwal Lee and fixed by Kihwal Lee <br>
|
|
|
+ <b>LeaseManager monitor thread can crash if the last block is complete but another block is not.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5557">HDFS-5557</a>.
|
|
|
+ Critical bug reported by Kihwal Lee and fixed by Kihwal Lee <br>
|
|
|
+ <b>Write pipeline recovery for the last packet in the block may cause rejection of valid replicas</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5552">HDFS-5552</a>.
|
|
|
+ Major bug reported by Shinichi Yamashita and fixed by Haohui Mai (namenode)<br>
|
|
|
+ <b>Fix wrong information of "Cluster summay" in dfshealth.html</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5548">HDFS-5548</a>.
|
|
|
+ Major improvement reported by Haohui Mai and fixed by Haohui Mai (nfs)<br>
|
|
|
+ <b>Use ConcurrentHashMap in portmap</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5545">HDFS-5545</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Allow specifying endpoints for listeners in HttpServer</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5544">HDFS-5544</a>.
|
|
|
+ Minor bug reported by sathish and fixed by sathish (hdfs-client)<br>
|
|
|
+ <b>Adding Test case For Checking dfs.checksum type as NULL value</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5540">HDFS-5540</a>.
|
|
|
+ Minor bug reported by Binglin Chang and fixed by Binglin Chang <br>
|
|
|
+ <b>Fix intermittent failure in TestBlocksWithNotEnoughRacks</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5538">HDFS-5538</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>URLConnectionFactory should pick up the SSL related configuration by default</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5536">HDFS-5536</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Implement HTTP policy for Namenode and DataNode</b><br>
|
|
|
+ <blockquote>Add new HTTP policy configuration. Users can use "dfs.http.policy" to control the HTTP endpoints for NameNode and DataNode. Specifically, The following values are supported:
|
|
|
+- HTTP_ONLY : Service is provided only on http
|
|
|
+- HTTPS_ONLY : Service is provided only on https
|
|
|
+- HTTP_AND_HTTPS : Service is provided both on http and https
|
|
|
+
|
|
|
+hadoop.ssl.enabled and dfs.https.enabled are deprecated. When the deprecated configuration properties are still configured, currently http policy is decided based on the following rules:
|
|
|
+1. If dfs.http.policy is set to HTTPS_ONLY or HTTP_AND_HTTPS. It picks the specified policy, otherwise it proceeds to 2~4.
|
|
|
+2. It picks HTTPS_ONLY if hadoop.ssl.enabled equals to true.
|
|
|
+3. It picks HTTP_AND_HTTPS if dfs.https.enable equals to true.
|
|
|
+4. It picks HTTP_ONLY for other configurations.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5533">HDFS-5533</a>.
|
|
|
+ Minor bug reported by Binglin Chang and fixed by Binglin Chang (snapshots)<br>
|
|
|
+ <b>Symlink delete/create should be treated as DELETE/CREATE in snapshot diff report</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5532">HDFS-5532</a>.
|
|
|
+ Major improvement reported by Vinayakumar B and fixed by Vinayakumar B (webhdfs)<br>
|
|
|
+ <b>Enable the webhdfs by default to support new HDFS web UI</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5526">HDFS-5526</a>.
|
|
|
+ Blocker bug reported by Tsz Wo (Nicholas), SZE and fixed by Kihwal Lee (datanode)<br>
|
|
|
+ <b>Datanode cannot roll back to previous layout version</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5525">HDFS-5525</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Inline dust templates</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5519">HDFS-5519</a>.
|
|
|
+ Minor sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>COMMIT handler should update the commit status after sync</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5514">HDFS-5514</a>.
|
|
|
+ Major sub-task reported by Daryn Sharp and fixed by Daryn Sharp (namenode)<br>
|
|
|
+ <b>FSNamesystem's fsLock should allow custom implementation</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5506">HDFS-5506</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Use URLConnectionFactory in DelegationTokenFetcher</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5504">HDFS-5504</a>.
|
|
|
+ Major bug reported by Vinayakumar B and fixed by Vinayakumar B (snapshots)<br>
|
|
|
+ <b>In HA mode, OP_DELETE_SNAPSHOT is not decrementing the safemode threshold, leads to NN safemode.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5502">HDFS-5502</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Fix HTTPS support in HsftpFileSystem</b><br>
|
|
|
+ <blockquote>Fix the https support in HsftpFileSystem. With the change the client now verifies the server certificate. In particular, client side will verify the Common Name of the certificate using a strategy specified by the configuration property "hadoop.ssl.hostname.verifier".</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5495">HDFS-5495</a>.
|
|
|
+ Major improvement reported by Andrew Wang and fixed by Jarek Jarcec Cecho <br>
|
|
|
+ <b>Remove further JUnit3 usages from HDFS</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5489">HDFS-5489</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Use TokenAspect in WebHDFSFileSystem</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5488">HDFS-5488</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Clean up TestHftpURLTimeout</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5487">HDFS-5487</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Introduce unit test for TokenAspect</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5476">HDFS-5476</a>.
|
|
|
+ Major bug reported by Jing Zhao and fixed by Jing Zhao <br>
|
|
|
+ <b>Snapshot: clean the blocks/files/directories under a renamed file/directory while deletion</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5474">HDFS-5474</a>.
|
|
|
+ Major bug reported by Uma Maheswara Rao G and fixed by sathish (snapshots)<br>
|
|
|
+ <b>Deletesnapshot can make Namenode in safemode on NN restarts.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5469">HDFS-5469</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>Add configuration property for the sub-directroy export path</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5467">HDFS-5467</a>.
|
|
|
+ Trivial improvement reported by Andrew Wang and fixed by Shinichi Yamashita <br>
|
|
|
+ <b>Remove tab characters in hdfs-default.xml</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5458">HDFS-5458</a>.
|
|
|
+ Major bug reported by Andrew Wang and fixed by Mike Mellenthin (datanode)<br>
|
|
|
+ <b>Datanode failed volume threshold ignored if exception is thrown in getDataDirsFromURIs</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5456">HDFS-5456</a>.
|
|
|
+ Critical bug reported by Chris Nauroth and fixed by Chris Nauroth (namenode)<br>
|
|
|
+ <b>NameNode startup progress creates new steps if caller attempts to create a counter for a step that doesn't already exist.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5454">HDFS-5454</a>.
|
|
|
+ Minor sub-task reported by Eric Sirianni and fixed by Arpit Agarwal (datanode)<br>
|
|
|
+ <b>DataNode UUID should be assigned prior to FsDataset initialization</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5449">HDFS-5449</a>.
|
|
|
+ Blocker bug reported by Kihwal Lee and fixed by Kihwal Lee <br>
|
|
|
+ <b>WebHdfs compatibility broken between 2.2 and 1.x / 23.x</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5444">HDFS-5444</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Choose default web UI based on browser capabilities</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5443">HDFS-5443</a>.
|
|
|
+ Major bug reported by Uma Maheswara Rao G and fixed by Jing Zhao (snapshots)<br>
|
|
|
+ <b>Delete 0-sized block when deleting an under-construction file that is included in snapshot</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5440">HDFS-5440</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Extract the logic of handling delegation tokens in HftpFileSystem to the TokenAspect class</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5438">HDFS-5438</a>.
|
|
|
+ Critical bug reported by Kihwal Lee and fixed by Kihwal Lee (namenode)<br>
|
|
|
+ <b>Flaws in block report processing can cause data loss</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5436">HDFS-5436</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Move HsFtpFileSystem and HFtpFileSystem into org.apache.hdfs.web</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5434">HDFS-5434</a>.
|
|
|
+ Minor bug reported by Buddy and fixed by (namenode)<br>
|
|
|
+ <b>Write resiliency for replica count 1</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5433">HDFS-5433</a>.
|
|
|
+ Critical bug reported by Aaron T. Myers and fixed by Aaron T. Myers (snapshots)<br>
|
|
|
+ <b>When reloading fsimage during checkpointing, we should clear existing snapshottable directories</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5432">HDFS-5432</a>.
|
|
|
+ Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (datanode , test)<br>
|
|
|
+ <b>TestDatanodeJsp fails on Windows due to assumption that loopback address resolves to host name localhost.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5428">HDFS-5428</a>.
|
|
|
+ Major bug reported by Vinayakumar B and fixed by Jing Zhao (snapshots)<br>
|
|
|
+ <b>under construction files deletion after snapshot+checkpoint+nn restart leads nn safemode</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5427">HDFS-5427</a>.
|
|
|
+ Major bug reported by Vinayakumar B and fixed by Vinayakumar B (snapshots)<br>
|
|
|
+ <b>not able to read deleted files from snapshot directly under snapshottable dir after checkpoint and NN restart</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5425">HDFS-5425</a>.
|
|
|
+ Major bug reported by sathish and fixed by Jing Zhao (namenode , snapshots)<br>
|
|
|
+ <b>Renaming underconstruction file with snapshots can make NN failure on restart</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5413">HDFS-5413</a>.
|
|
|
+ Major bug reported by Chris Nauroth and fixed by Chris Nauroth (scripts)<br>
|
|
|
+ <b>hdfs.cmd does not support passthrough to any arbitrary class.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5407">HDFS-5407</a>.
|
|
|
+ Trivial bug reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Fix typos in DFSClientCache</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5406">HDFS-5406</a>.
|
|
|
+ Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)<br>
|
|
|
+ <b>Send incremental block reports for all storages in a single call</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5403">HDFS-5403</a>.
|
|
|
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (webhdfs)<br>
|
|
|
+ <b>WebHdfs client cannot communicate with older WebHdfs servers post HDFS-5306</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5400">HDFS-5400</a>.
|
|
|
+ Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe <br>
|
|
|
+ <b>DFS_CLIENT_MMAP_CACHE_THREAD_RUNS_PER_TIMEOUT constant is set to the wrong value</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5399">HDFS-5399</a>.
|
|
|
+ Major improvement reported by Jing Zhao and fixed by Jing Zhao <br>
|
|
|
+ <b>Revisit SafeModeException and corresponding retry policies</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5393">HDFS-5393</a>.
|
|
|
+ Minor sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Serve bootstrap and jQuery locally</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5382">HDFS-5382</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Implement the UI of browsing filesystems in HTML 5 page</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5379">HDFS-5379</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Update links to datanode information in dfshealth.html</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5375">HDFS-5375</a>.
|
|
|
+ Minor bug reported by Chris Nauroth and fixed by Chris Nauroth (tools)<br>
|
|
|
+ <b>hdfs.cmd does not expose several snapshot commands.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5374">HDFS-5374</a>.
|
|
|
+ Trivial bug reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
|
|
|
+ <b>Remove deadcode in DFSOutputStream</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5372">HDFS-5372</a>.
|
|
|
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Vinayakumar B (namenode)<br>
|
|
|
+ <b>In FSNamesystem, hasReadLock() returns false if the current thread holds the write lock</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5371">HDFS-5371</a>.
|
|
|
+ Minor improvement reported by Jing Zhao and fixed by Jing Zhao (ha , test)<br>
|
|
|
+ <b>Let client retry the same NN when "dfs.client.test.drop.namenode.response.number" is enabled</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5370">HDFS-5370</a>.
|
|
|
+ Trivial bug reported by Kousuke Saruta and fixed by Kousuke Saruta (hdfs-client)<br>
|
|
|
+ <b>Typo in Error Message: different between range in condition and range in error message</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5365">HDFS-5365</a>.
|
|
|
+ Major bug reported by Radim Kolar and fixed by Radim Kolar (build , libhdfs)<br>
|
|
|
+ <b>Fix libhdfs compile error on FreeBSD9</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5364">HDFS-5364</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>Add OpenFileCtx cache</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5363">HDFS-5363</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Refactor WebHdfsFileSystem: move SPENGO-authenticated connection creation to URLConnectionFactory </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5360">HDFS-5360</a>.
|
|
|
+ Minor improvement reported by Shinichi Yamashita and fixed by Shinichi Yamashita (snapshots)<br>
|
|
|
+ <b>Improvement of usage message of renameSnapshot and deleteSnapshot</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5353">HDFS-5353</a>.
|
|
|
+ Blocker bug reported by Haohui Mai and fixed by Colin Patrick McCabe <br>
|
|
|
+ <b>Short circuit reads fail when dfs.encrypt.data.transfer is enabled</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5352">HDFS-5352</a>.
|
|
|
+ Minor bug reported by Ted Yu and fixed by Ted Yu <br>
|
|
|
+ <b>Server#initLog() doesn't close InputStream in httpfs</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5350">HDFS-5350</a>.
|
|
|
+ Minor improvement reported by Rob Weltman and fixed by Jimmy Xiang (namenode)<br>
|
|
|
+ <b>Name Node should report fsimage transfer time as a metric</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5347">HDFS-5347</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (documentation)<br>
|
|
|
+ <b>add HDFS NFS user guide</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5346">HDFS-5346</a>.
|
|
|
+ Major bug reported by Kihwal Lee and fixed by Ravi Prakash (namenode , performance)<br>
|
|
|
+ <b>Avoid unnecessary call to getNumLiveDataNodes() for each block during IBR processing</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5344">HDFS-5344</a>.
|
|
|
+ Minor improvement reported by sathish and fixed by sathish (snapshots , tools)<br>
|
|
|
+ <b>Make LsSnapshottableDir as Tool interface implementation</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5343">HDFS-5343</a>.
|
|
|
+ Major bug reported by sathish and fixed by sathish (hdfs-client)<br>
|
|
|
+ <b>When cat command is issued on snapshot files getting unexpected result</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5342">HDFS-5342</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Provide more information in the FSNamesystem JMX interfaces</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5341">HDFS-5341</a>.
|
|
|
+ Major bug reported by qus-jiawei and fixed by qus-jiawei (datanode)<br>
|
|
|
+ <b>Reduce fsdataset lock duration during directory scanning.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5338">HDFS-5338</a>.
|
|
|
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (namenode)<br>
|
|
|
+ <b>Add a conf to disable hostname check in DN registration</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5337">HDFS-5337</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>should do hsync for a commit request even there is no pending writes</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5336">HDFS-5336</a>.
|
|
|
+ Minor bug reported by Akira AJISAKA and fixed by Akira AJISAKA (namenode)<br>
|
|
|
+ <b>DataNode should not output 'StartupProgress' metrics</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5335">HDFS-5335</a>.
|
|
|
+ Major bug reported by Arpit Gupta and fixed by Haohui Mai <br>
|
|
|
+ <b>DFSOutputStream#close() keeps throwing exceptions when it is called multiple times</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5334">HDFS-5334</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Implement dfshealth.jsp in HTML pages</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5331">HDFS-5331</a>.
|
|
|
+ Major improvement reported by Vinayakumar B and fixed by Vinayakumar B (snapshots)<br>
|
|
|
+ <b>make SnapshotDiff.java to a o.a.h.util.Tool interface implementation</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5330">HDFS-5330</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>fix readdir and readdirplus for large directories</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5329">HDFS-5329</a>.
|
|
|
+ Major bug reported by Brandon Li and fixed by Brandon Li (namenode , nfs)<br>
|
|
|
+ <b>Update FSNamesystem#getListing() to handle inode path in startAfter token</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5325">HDFS-5325</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Remove WebHdfsFileSystem#ConnRunner</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5323">HDFS-5323</a>.
|
|
|
+ Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (namenode)<br>
|
|
|
+ <b>Remove some deadcode in BlockManager</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5322">HDFS-5322</a>.
|
|
|
+ Major bug reported by Arpit Gupta and fixed by Jing Zhao (ha)<br>
|
|
|
+ <b>HDFS delegation token not found in cache errors seen on secure HA clusters</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5317">HDFS-5317</a>.
|
|
|
+ Critical sub-task reported by Suresh Srinivas and fixed by Haohui Mai <br>
|
|
|
+ <b>Go back to DFS Home link does not work on datanode webUI</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5316">HDFS-5316</a>.
|
|
|
+ Critical sub-task reported by Suresh Srinivas and fixed by Haohui Mai <br>
|
|
|
+ <b>Namenode ignores the default https port</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5312">HDFS-5312</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Generate HTTP / HTTPS URL in DFSUtil#getInfoServer() based on the configured http policy</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5307">HDFS-5307</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Support both HTTP and HTTPS in jsp pages</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5305">HDFS-5305</a>.
|
|
|
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
|
|
|
+ <b>Add https support in HDFS</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5297">HDFS-5297</a>.
|
|
|
+ Major bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)<br>
|
|
|
+ <b>Fix dead links in HDFS site documents</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5291">HDFS-5291</a>.
|
|
|
+ Critical bug reported by Arpit Gupta and fixed by Jing Zhao (ha)<br>
|
|
|
+ <b>Clients need to retry when Active NN is in SafeMode</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5288">HDFS-5288</a>.
|
|
|
+ Major sub-task reported by Haohui Mai and fixed by Haohui Mai (nfs)<br>
|
|
|
+ <b>Close idle connections in portmap</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5283">HDFS-5283</a>.
|
|
|
+ Critical bug reported by Vinayakumar B and fixed by Vinayakumar B (snapshots)<br>
|
|
|
+ <b>NN not coming out of startup safemode due to under construction blocks only inside snapshots also counted in safemode threshhold</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5281">HDFS-5281</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>COMMIT request should not block</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5276">HDFS-5276</a>.
|
|
|
+ Major bug reported by Chengxiang Li and fixed by Colin Patrick McCabe <br>
|
|
|
+ <b>FileSystem.Statistics got performance issue on multi-thread read/write.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5267">HDFS-5267</a>.
|
|
|
+ Minor improvement reported by Junping Du and fixed by Junping Du <br>
|
|
|
+ <b>Remove volatile from LightWeightHashSet</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5260">HDFS-5260</a>.
|
|
|
+ Major new feature reported by Chris Nauroth and fixed by Chris Nauroth (hdfs-client , libhdfs)<br>
|
|
|
+ <b>Merge zero-copy memory-mapped HDFS client reads to trunk and branch-2.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5257">HDFS-5257</a>.
|
|
|
+ Major bug reported by Vinayakumar B and fixed by Vinayakumar B (hdfs-client , namenode)<br>
|
|
|
+ <b>addBlock() retry should return LocatedBlock with locations else client will get AIOBE</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5252">HDFS-5252</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>Stable write is not handled correctly in someplace</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5240">HDFS-5240</a>.
|
|
|
+ Major sub-task reported by Daryn Sharp and fixed by Daryn Sharp (namenode)<br>
|
|
|
+ <b>Separate formatting from logging in the audit logger API</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5239">HDFS-5239</a>.
|
|
|
+ Major sub-task reported by Daryn Sharp and fixed by Daryn Sharp (namenode)<br>
|
|
|
+ <b>Allow FSNamesystem lock fairness to be configurable</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5220">HDFS-5220</a>.
|
|
|
+ Major improvement reported by Rob Weltman and fixed by Jimmy Xiang (namenode)<br>
|
|
|
+ <b>Expose group resolution time as metric</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5207">HDFS-5207</a>.
|
|
|
+ Major improvement reported by Junping Du and fixed by Junping Du (namenode)<br>
|
|
|
+ <b>In BlockPlacementPolicy, update 2 parameters of chooseTarget()</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5188">HDFS-5188</a>.
|
|
|
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (namenode)<br>
|
|
|
+ <b>Clean up BlockPlacementPolicy and its implementations</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5171">HDFS-5171</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Haohui Mai (nfs)<br>
|
|
|
+ <b>NFS should create input stream for a file and try to share it with multiple read requests</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5170">HDFS-5170</a>.
|
|
|
+ Trivial bug reported by Andrew Wang and fixed by Andrew Wang <br>
|
|
|
+ <b>BlockPlacementPolicyDefault uses the wrong classname when alerting to enable debug logging</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5164">HDFS-5164</a>.
|
|
|
+ Minor bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (namenode)<br>
|
|
|
+ <b>deleteSnapshot should check if OperationCategory.WRITE is possible before taking write lock</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5144">HDFS-5144</a>.
|
|
|
+ Minor improvement reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)<br>
|
|
|
+ <b>Document time unit to NameNodeMetrics.java</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5136">HDFS-5136</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>MNT EXPORT should give the full group list which can mount the exports</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5130">HDFS-5130</a>.
|
|
|
+ Minor test reported by Binglin Chang and fixed by Binglin Chang (test)<br>
|
|
|
+ <b>Add test for snapshot related FsShell and DFSAdmin commands</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5122">HDFS-5122</a>.
|
|
|
+ Major bug reported by Arpit Gupta and fixed by Haohui Mai (ha , webhdfs)<br>
|
|
|
+ <b>Support failover and retry in WebHdfsFileSystem for NN HA</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5110">HDFS-5110</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>Change FSDataOutputStream to HdfsDataOutputStream for opened streams to fix type cast error</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5107">HDFS-5107</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>Fix array copy error in Readdir and Readdirplus responses</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5104">HDFS-5104</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>Support dotdot name in NFS LOOKUP operation</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5093">HDFS-5093</a>.
|
|
|
+ Minor bug reported by Chuan Liu and fixed by Chuan Liu (test)<br>
|
|
|
+ <b>TestGlobPaths should re-use the MiniDFSCluster to avoid failure on Windows</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5078">HDFS-5078</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>Support file append in NFSv3 gateway to enable data streaming to HDFS</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5075">HDFS-5075</a>.
|
|
|
+ Major bug reported by Timothy St. Clair and fixed by Timothy St. Clair <br>
|
|
|
+ <b>httpfs-config.sh calls out incorrect env script name</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5074">HDFS-5074</a>.
|
|
|
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon (ha , namenode)<br>
|
|
|
+ <b>Allow starting up from an fsimage checkpoint in the middle of a segment</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5073">HDFS-5073</a>.
|
|
|
+ Minor bug reported by Kihwal Lee and fixed by Arpit Agarwal (test)<br>
|
|
|
+ <b>TestListCorruptFileBlocks fails intermittently </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5071">HDFS-5071</a>.
|
|
|
+ Major sub-task reported by Kihwal Lee and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>Change hdfs-nfs parent project to hadoop-project</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5069">HDFS-5069</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>Include hadoop-nfs and hadoop-hdfs-nfs into hadoop dist for NFS deployment</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5068">HDFS-5068</a>.
|
|
|
+ Major improvement reported by Konstantin Shvachko and fixed by Konstantin Shvachko (benchmarks)<br>
|
|
|
+ <b>Convert NNThroughputBenchmark to a Tool to allow generic options.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5065">HDFS-5065</a>.
|
|
|
+ Major bug reported by Ivan Mitic and fixed by Ivan Mitic (hdfs-client , test)<br>
|
|
|
+ <b>TestSymlinkHdfsDisable fails on Windows</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5043">HDFS-5043</a>.
|
|
|
+ Major bug reported by Brandon Li and fixed by Brandon Li <br>
|
|
|
+ <b>For HdfsFileStatus, set default value of childrenNum to -1 instead of 0 to avoid confusing applications</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5037">HDFS-5037</a>.
|
|
|
+ Critical improvement reported by Todd Lipcon and fixed by Andrew Wang (ha , namenode)<br>
|
|
|
+ <b>Active NN should trigger its own edit log rolls</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5035">HDFS-5035</a>.
|
|
|
+ Major bug reported by Andrew Wang and fixed by Andrew Wang (namenode)<br>
|
|
|
+ <b>getFileLinkStatus and rename do not correctly check permissions of symlinks</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5034">HDFS-5034</a>.
|
|
|
+ Trivial improvement reported by Andrew Wang and fixed by Andrew Wang (namenode)<br>
|
|
|
+ <b>Remove debug prints from getFileLinkInfo</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5023">HDFS-5023</a>.
|
|
|
+ Major bug reported by Ravi Prakash and fixed by Mit Desai (snapshots , test)<br>
|
|
|
+ <b>TestSnapshotPathINodes.testAllowSnapshot is failing with jdk7</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5014">HDFS-5014</a>.
|
|
|
+ Major bug reported by Vinayakumar B and fixed by Vinayakumar B (datanode , ha)<br>
|
|
|
+ <b>BPOfferService#processCommandFromActor() synchronization on namenode RPC call delays IBR to Active NN, if Stanby NN is unstable</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-5004">HDFS-5004</a>.
|
|
|
+ Major improvement reported by Trevor Lorimer and fixed by Trevor Lorimer (namenode)<br>
|
|
|
+ <b>Add additional JMX bean for NameNode status data</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4997">HDFS-4997</a>.
|
|
|
+ Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (libhdfs)<br>
|
|
|
+ <b>libhdfs doesn't return correct error codes in most cases</b><br>
|
|
|
+ <blockquote>libhdfs now returns correct codes in errno. Previously, due to a bug, many functions set errno to 255 instead of the more specific error code.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4995">HDFS-4995</a>.
|
|
|
+ Major bug reported by Kihwal Lee and fixed by Kihwal Lee (namenode)<br>
|
|
|
+ <b>Make getContentSummary() less expensive</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4994">HDFS-4994</a>.
|
|
|
+ Minor bug reported by Kihwal Lee and fixed by Robert Parker (namenode)<br>
|
|
|
+ <b>Audit log getContentSummary() calls</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4983">HDFS-4983</a>.
|
|
|
+ Major improvement reported by Harsh J and fixed by Yongjun Zhang (webhdfs)<br>
|
|
|
+ <b>Numeric usernames do not work with WebHDFS FS</b><br>
|
|
|
+ <blockquote>Add a new configuration property "dfs.webhdfs.user.provider.user.pattern" for specifying user name filters for WebHDFS.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4962">HDFS-4962</a>.
|
|
|
+ Minor sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (nfs)<br>
|
|
|
+ <b>Use enum for nfs constants</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4949">HDFS-4949</a>.
|
|
|
+ Major new feature reported by Andrew Wang and fixed by Andrew Wang (datanode , namenode)<br>
|
|
|
+ <b>Centralized cache management in HDFS</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4948">HDFS-4948</a>.
|
|
|
+ Major bug reported by Robert Joseph Evans and fixed by Brandon Li <br>
|
|
|
+ <b>mvn site for hadoop-hdfs-nfs fails</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4947">HDFS-4947</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Jing Zhao (nfs)<br>
|
|
|
+ <b>Add NFS server export table to control export by hostname or IP range</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4885">HDFS-4885</a>.
|
|
|
+ Major sub-task reported by Junping Du and fixed by Junping Du <br>
|
|
|
+ <b>Update verifyBlockPlacement() API in BlockPlacementPolicy</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4879">HDFS-4879</a>.
|
|
|
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (namenode)<br>
|
|
|
+ <b>Add "blocked ArrayList" collection to avoid CMS full GCs</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4860">HDFS-4860</a>.
|
|
|
+ Major improvement reported by Trevor Lorimer and fixed by Trevor Lorimer (namenode)<br>
|
|
|
+ <b>Add additional attributes to JMX beans</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4816">HDFS-4816</a>.
|
|
|
+ Major bug reported by Andrew Wang and fixed by Andrew Wang (namenode)<br>
|
|
|
+ <b>transitionToActive blocks if the SBN is doing checkpoint image transfer</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4772">HDFS-4772</a>.
|
|
|
+ Minor improvement reported by Brandon Li and fixed by Brandon Li (namenode)<br>
|
|
|
+ <b>Add number of children in HdfsFileStatus</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4763">HDFS-4763</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>Add script changes/utility for starting NFS gateway</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4762">HDFS-4762</a>.
|
|
|
+ Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>Provide HDFS based NFSv3 and Mountd implementation</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4657">HDFS-4657</a>.
|
|
|
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (namenode)<br>
|
|
|
+ <b>Limit the number of blocks logged by the NN after a block report to a configurable value.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4633">HDFS-4633</a>.
|
|
|
+ Major bug reported by Chris Nauroth and fixed by Chris Nauroth (hdfs-client , test)<br>
|
|
|
+ <b>TestDFSClientExcludedNodes fails sporadically if excluded nodes cache expires too quickly</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4517">HDFS-4517</a>.
|
|
|
+ Major test reported by Vadim Bondarev and fixed by Ivan A. Veselovsky <br>
|
|
|
+ <b>Cover class RemoteBlockReader with unit tests</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4516">HDFS-4516</a>.
|
|
|
+ Critical bug reported by Uma Maheswara Rao G and fixed by Vinayakumar B (namenode)<br>
|
|
|
+ <b>Client crash after block allocation and NN switch before lease recovery for the same file can cause readers to fail forever</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4512">HDFS-4512</a>.
|
|
|
+ Major test reported by Vadim Bondarev and fixed by Vadim Bondarev <br>
|
|
|
+ <b>Cover package org.apache.hadoop.hdfs.server.common with tests</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4511">HDFS-4511</a>.
|
|
|
+ Major test reported by Vadim Bondarev and fixed by Andrey Klochkov <br>
|
|
|
+ <b>Cover package org.apache.hadoop.hdfs.tools with unit test</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4510">HDFS-4510</a>.
|
|
|
+ Major test reported by Vadim Bondarev and fixed by Andrey Klochkov <br>
|
|
|
+ <b>Cover classes ClusterJspHelper/NamenodeJspHelper with unit tests</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4491">HDFS-4491</a>.
|
|
|
+ Major test reported by Tsuyoshi OZAWA and fixed by Andrey Klochkov (test)<br>
|
|
|
+ <b>Parallel testing HDFS</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4376">HDFS-4376</a>.
|
|
|
+ Major bug reported by Aaron T. Myers and fixed by Junping Du (balancer)<br>
|
|
|
+ <b> Fix several race conditions in Balancer and resolve intermittent timeout of TestBalancerWithNodeGroup</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4329">HDFS-4329</a>.
|
|
|
+ Major bug reported by Andy Isaacson and fixed by Cristina L. Abad (hdfs-client)<br>
|
|
|
+ <b>DFSShell issues with directories with spaces in name</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4278">HDFS-4278</a>.
|
|
|
+ Major improvement reported by Harsh J and fixed by Kousuke Saruta (datanode , namenode)<br>
|
|
|
+ <b>Log an ERROR when DFS_BLOCK_ACCESS_TOKEN_ENABLE config is disabled but security is turned on.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4201">HDFS-4201</a>.
|
|
|
+ Critical bug reported by Eli Collins and fixed by Jimmy Xiang (namenode)<br>
|
|
|
+ <b>NPE in BPServiceActor#sendHeartBeat</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4096">HDFS-4096</a>.
|
|
|
+ Major sub-task reported by Jing Zhao and fixed by Haohui Mai (datanode , namenode)<br>
|
|
|
+ <b>Add snapshot information to namenode WebUI</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3987">HDFS-3987</a>.
|
|
|
+ Major sub-task reported by Alejandro Abdelnur and fixed by Haohui Mai <br>
|
|
|
+ <b>Support webhdfs over HTTPS</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3981">HDFS-3981</a>.
|
|
|
+ Major bug reported by Xiaobo Peng and fixed by Xiaobo Peng (namenode)<br>
|
|
|
+ <b>access time is set without holding FSNamesystem write lock</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3934">HDFS-3934</a>.
|
|
|
+ Minor bug reported by Andy Isaacson and fixed by Colin Patrick McCabe <br>
|
|
|
+ <b>duplicative dfs_hosts entries handled wrong</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2933">HDFS-2933</a>.
|
|
|
+ Major improvement reported by Philip Zeyliger and fixed by Vivek Ganesan (datanode)<br>
|
|
|
+ <b>Improve DataNode Web UI Index Page</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10317">HADOOP-10317</a>.
|
|
|
+ Major bug reported by Andrew Wang and fixed by Andrew Wang <br>
|
|
|
+ <b>Rename branch-2.3 release version from 2.4.0-SNAPSHOT to 2.3.0-SNAPSHOT</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10313">HADOOP-10313</a>.
|
|
|
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
|
|
|
+ <b>Script and jenkins job to produce Hadoop release artifacts</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10311">HADOOP-10311</a>.
|
|
|
+ Blocker bug reported by Suresh Srinivas and fixed by Alejandro Abdelnur <br>
|
|
|
+ <b>Cleanup vendor names from the code base</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10310">HADOOP-10310</a>.
|
|
|
+ Blocker bug reported by Aaron T. Myers and fixed by Aaron T. Myers (security)<br>
|
|
|
+ <b>SaslRpcServer should be initialized even when no secret manager present</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10305">HADOOP-10305</a>.
|
|
|
+ Major bug reported by Akira AJISAKA and fixed by Akira AJISAKA (metrics)<br>
|
|
|
+ <b>Add "rpc.metrics.quantile.enable" and "rpc.metrics.percentiles.intervals" to core-default.xml</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10292">HADOOP-10292</a>.
|
|
|
+ Major bug reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Restore HttpServer from branch-2.2 in branch-2</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10291">HADOOP-10291</a>.
|
|
|
+ Major bug reported by Mit Desai and fixed by Mit Desai <br>
|
|
|
+ <b>TestSecurityUtil#testSocketAddrWithIP fails</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10288">HADOOP-10288</a>.
|
|
|
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon (util)<br>
|
|
|
+ <b>Explicit reference to Log4JLogger breaks non-log4j users</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10274">HADOOP-10274</a>.
|
|
|
+ Minor improvement reported by takeshi.miao and fixed by takeshi.miao (security)<br>
|
|
|
+ <b>Lower the logging level from ERROR to WARN for UGI.doAs method</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10273">HADOOP-10273</a>.
|
|
|
+ Major bug reported by Arpit Agarwal and fixed by Arpit Agarwal (build)<br>
|
|
|
+ <b>Fix 'mvn site'</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10255">HADOOP-10255</a>.
|
|
|
+ Blocker bug reported by Haohui Mai and fixed by Haohui Mai <br>
|
|
|
+ <b>Rename HttpServer to HttpServer2 to retain older HttpServer in branch-2 for compatibility</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10252">HADOOP-10252</a>.
|
|
|
+ Major bug reported by Jimmy Xiang and fixed by Jimmy Xiang <br>
|
|
|
+ <b>HttpServer can't start if hostname is not specified</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10250">HADOOP-10250</a>.
|
|
|
+ Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang <br>
|
|
|
+ <b>VersionUtil returns wrong value when comparing two versions</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10248">HADOOP-10248</a>.
|
|
|
+ Major improvement reported by Ted Yu and fixed by Akira AJISAKA <br>
|
|
|
+ <b>Property name should be included in the exception where property value is null</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10240">HADOOP-10240</a>.
|
|
|
+ Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (documentation)<br>
|
|
|
+ <b>Windows build instructions incorrectly state requirement of protoc 2.4.1 instead of 2.5.0</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10236">HADOOP-10236</a>.
|
|
|
+ Trivial bug reported by Akira AJISAKA and fixed by Akira AJISAKA <br>
|
|
|
+ <b>Fix typo in o.a.h.ipc.Client#checkResponse</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10235">HADOOP-10235</a>.
|
|
|
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
|
|
|
+ <b>Hadoop tarball has 2 versions of stax-api JARs</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10234">HADOOP-10234</a>.
|
|
|
+ Major bug reported by Chris Nauroth and fixed by Chris Nauroth (scripts)<br>
|
|
|
+ <b>"hadoop.cmd jar" does not propagate exit code.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10228">HADOOP-10228</a>.
|
|
|
+ Minor improvement reported by Haohui Mai and fixed by Haohui Mai (fs)<br>
|
|
|
+ <b>FsPermission#fromShort() should cache FsAction.values()</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10223">HADOOP-10223</a>.
|
|
|
+ Minor bug reported by Ted Yu and fixed by Ted Yu <br>
|
|
|
+ <b>MiniKdc#main() should close the FileReader it creates</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10214">HADOOP-10214</a>.
|
|
|
+ Major bug reported by Liang Xie and fixed by Liang Xie (ha)<br>
|
|
|
+ <b>Fix multithreaded correctness warnings in ActiveStandbyElector</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10212">HADOOP-10212</a>.
|
|
|
+ Major bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)<br>
|
|
|
+ <b>Incorrect compile command in Native Library document</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10208">HADOOP-10208</a>.
|
|
|
+ Trivial improvement reported by Benoy Antony and fixed by Benoy Antony <br>
|
|
|
+ <b>Remove duplicate initialization in StringUtils.getStringCollection</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10207">HADOOP-10207</a>.
|
|
|
+ Minor test reported by Jimmy Xiang and fixed by Jimmy Xiang <br>
|
|
|
+ <b>TestUserGroupInformation#testLogin is flaky</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10203">HADOOP-10203</a>.
|
|
|
+ Major bug reported by Andrei Savu and fixed by Andrei Savu (fs/s3)<br>
|
|
|
+ <b>Connection leak in Jets3tNativeFileSystemStore#retrieveMetadata </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10198">HADOOP-10198</a>.
|
|
|
+ Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (native)<br>
|
|
|
+ <b>DomainSocket: add support for socketpair</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10193">HADOOP-10193</a>.
|
|
|
+ Minor bug reported by Gregory Chanan and fixed by Gregory Chanan (security)<br>
|
|
|
+ <b>hadoop-auth's PseudoAuthenticationHandler can consume getInputStream</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10178">HADOOP-10178</a>.
|
|
|
+ Major bug reported by shanyu zhao and fixed by shanyu zhao (conf)<br>
|
|
|
+ <b>Configuration deprecation always emit "deprecated" warnings when a new key is used</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10175">HADOOP-10175</a>.
|
|
|
+ Major bug reported by Chuan Liu and fixed by Chuan Liu (fs)<br>
|
|
|
+ <b>Har files system authority should preserve userinfo</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10173">HADOOP-10173</a>.
|
|
|
+ Critical improvement reported by Daryn Sharp and fixed by Daryn Sharp (ipc)<br>
|
|
|
+ <b>Remove UGI from DIGEST-MD5 SASL server creation</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10172">HADOOP-10172</a>.
|
|
|
+ Critical improvement reported by Daryn Sharp and fixed by Daryn Sharp (ipc)<br>
|
|
|
+ <b>Cache SASL server factories</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10171">HADOOP-10171</a>.
|
|
|
+ Major bug reported by Mit Desai and fixed by Mit Desai <br>
|
|
|
+ <b>TestRPC fails intermittently on jkd7</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10169">HADOOP-10169</a>.
|
|
|
+ Minor improvement reported by Liang Xie and fixed by Liang Xie (metrics)<br>
|
|
|
+ <b>remove the unnecessary synchronized in JvmMetrics class</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10168">HADOOP-10168</a>.
|
|
|
+ Major bug reported by Thejas M Nair and fixed by Thejas M Nair <br>
|
|
|
+ <b>fix javadoc of ReflectionUtils.copy </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10167">HADOOP-10167</a>.
|
|
|
+ Major improvement reported by Mikhail Antonov and fixed by (build)<br>
|
|
|
+ <b>Mark hadoop-common source as UTF-8 in Maven pom files / refactoring</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10164">HADOOP-10164</a>.
|
|
|
+ Major improvement reported by Robert Joseph Evans and fixed by Robert Joseph Evans <br>
|
|
|
+ <b>Allow UGI to login with a known Subject</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10162">HADOOP-10162</a>.
|
|
|
+ Major bug reported by Mit Desai and fixed by Mit Desai <br>
|
|
|
+ <b>Fix symlink-related test failures in TestFileContextResolveAfs and TestStat in branch-2</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10147">HADOOP-10147</a>.
|
|
|
+ Minor bug reported by Eric Sirianni and fixed by Steve Loughran (build)<br>
|
|
|
+ <b>Upgrade to commons-logging 1.1.3 to avoid potential deadlock in MiniDFSCluster</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10146">HADOOP-10146</a>.
|
|
|
+ Critical bug reported by Daryn Sharp and fixed by Daryn Sharp (util)<br>
|
|
|
+ <b>Workaround JDK7 Process fd close bug</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10143">HADOOP-10143</a>.
|
|
|
+ Major improvement reported by Liang Xie and fixed by Liang Xie (io)<br>
|
|
|
+ <b>replace WritableFactories's hashmap with ConcurrentHashMap</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10142">HADOOP-10142</a>.
|
|
|
+ Major bug reported by Vinayakumar B and fixed by Vinayakumar B <br>
|
|
|
+ <b>Avoid groups lookup for unprivileged users such as "dr.who"</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10135">HADOOP-10135</a>.
|
|
|
+ Major bug reported by David Dobbins and fixed by David Dobbins (fs)<br>
|
|
|
+ <b>writes to swift fs over partition size leave temp files and empty output file</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10132">HADOOP-10132</a>.
|
|
|
+ Minor improvement reported by Ted Yu and fixed by Ted Yu <br>
|
|
|
+ <b>RPC#stopProxy() should log the class of proxy when IllegalArgumentException is encountered</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10130">HADOOP-10130</a>.
|
|
|
+ Minor bug reported by Binglin Chang and fixed by Binglin Chang <br>
|
|
|
+ <b>RawLocalFS::LocalFSFileInputStream.pread does not track FS::Statistics</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10129">HADOOP-10129</a>.
|
|
|
+ Critical bug reported by Daryn Sharp and fixed by Daryn Sharp (tools/distcp)<br>
|
|
|
+ <b>Distcp may succeed when it fails</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10127">HADOOP-10127</a>.
|
|
|
+ Major bug reported by Karthik Kambatla and fixed by Karthik Kambatla (ipc)<br>
|
|
|
+ <b>Add ipc.client.connect.retry.interval to control the frequency of connection retries</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10126">HADOOP-10126</a>.
|
|
|
+ Minor bug reported by Vinayakumar B and fixed by Vinayakumar B (util)<br>
|
|
|
+ <b>LightWeightGSet log message is confusing : "2.0% max memory = 2.0 GB"</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10125">HADOOP-10125</a>.
|
|
|
+ Major bug reported by Ming Ma and fixed by Ming Ma (ipc)<br>
|
|
|
+ <b>no need to process RPC request if the client connection has been dropped</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10112">HADOOP-10112</a>.
|
|
|
+ Major bug reported by Brandon Li and fixed by Brandon Li (tools)<br>
|
|
|
+ <b>har file listing doesn't work with wild card</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10111">HADOOP-10111</a>.
|
|
|
+ Major improvement reported by Kihwal Lee and fixed by Kihwal Lee <br>
|
|
|
+ <b>Allow DU to be initialized with an initial value</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10110">HADOOP-10110</a>.
|
|
|
+ Blocker bug reported by Chuan Liu and fixed by Chuan Liu (build)<br>
|
|
|
+ <b>hadoop-auth has a build break due to missing dependency</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10109">HADOOP-10109</a>.
|
|
|
+ Major sub-task reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (test)<br>
|
|
|
+ <b>Fix test failure in TestOfflineEditsViewer introduced by HADOOP-10052</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10107">HADOOP-10107</a>.
|
|
|
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Kihwal Lee (ipc)<br>
|
|
|
+ <b>Server.getNumOpenConnections may throw NPE</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10106">HADOOP-10106</a>.
|
|
|
+ Minor bug reported by Ming Ma and fixed by Ming Ma <br>
|
|
|
+ <b>Incorrect thread name in RPC log messages</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10103">HADOOP-10103</a>.
|
|
|
+ Minor sub-task reported by Steve Loughran and fixed by Akira AJISAKA (build)<br>
|
|
|
+ <b>update commons-lang to 2.6</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10102">HADOOP-10102</a>.
|
|
|
+ Minor sub-task reported by Steve Loughran and fixed by Akira AJISAKA (build)<br>
|
|
|
+ <b>update commons IO from 2.1 to 2.4</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10100">HADOOP-10100</a>.
|
|
|
+ Major bug reported by Robert Kanter and fixed by Robert Kanter <br>
|
|
|
+ <b>MiniKDC shouldn't use apacheds-all artifact</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10095">HADOOP-10095</a>.
|
|
|
+ Minor improvement reported by Nicolas Liochon and fixed by Nicolas Liochon (io)<br>
|
|
|
+ <b>Performance improvement in CodecPool</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10094">HADOOP-10094</a>.
|
|
|
+ Trivial bug reported by Enis Soztutar and fixed by Enis Soztutar (util)<br>
|
|
|
+ <b>NPE in GenericOptionsParser#preProcessForWindows()</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10093">HADOOP-10093</a>.
|
|
|
+ Major bug reported by shanyu zhao and fixed by shanyu zhao (conf)<br>
|
|
|
+ <b>hadoop-env.cmd sets HADOOP_CLIENT_OPTS with a max heap size that is too small.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10090">HADOOP-10090</a>.
|
|
|
+ Major bug reported by Ivan Mitic and fixed by Ivan Mitic (metrics)<br>
|
|
|
+ <b>Jobtracker metrics not updated properly after execution of a mapreduce job</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10088">HADOOP-10088</a>.
|
|
|
+ Major bug reported by Raja Aluri and fixed by Raja Aluri (build)<br>
|
|
|
+ <b>copy-nativedistlibs.sh needs to quote snappy lib dir</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10087">HADOOP-10087</a>.
|
|
|
+ Major bug reported by Yu Gao and fixed by Colin Patrick McCabe (security)<br>
|
|
|
+ <b>UserGroupInformation.getGroupNames() fails to return primary group first when JniBasedUnixGroupsMappingWithFallback is used</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10086">HADOOP-10086</a>.
|
|
|
+ Minor improvement reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation)<br>
|
|
|
+ <b>User document for authentication in secure cluster</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10081">HADOOP-10081</a>.
|
|
|
+ Critical bug reported by Jason Lowe and fixed by Tsuyoshi OZAWA (ipc)<br>
|
|
|
+ <b>Client.setupIOStreams can leak socket resources on exception or error</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10079">HADOOP-10079</a>.
|
|
|
+ Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe <br>
|
|
|
+ <b>log a warning message if group resolution takes too long.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10078">HADOOP-10078</a>.
|
|
|
+ Minor bug reported by Robert Kanter and fixed by Robert Kanter (security)<br>
|
|
|
+ <b>KerberosAuthenticator always does SPNEGO</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10072">HADOOP-10072</a>.
|
|
|
+ Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (nfs , test)<br>
|
|
|
+ <b>TestNfsExports#testMultiMatchers fails due to non-deterministic timing around cache expiry check.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10067">HADOOP-10067</a>.
|
|
|
+ Minor improvement reported by Robert Rati and fixed by Robert Rati <br>
|
|
|
+ <b>Missing POM dependency on jsr305</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10064">HADOOP-10064</a>.
|
|
|
+ Major improvement reported by Arpit Agarwal and fixed by Arpit Agarwal (build)<br>
|
|
|
+ <b>Upgrade to maven antrun plugin version 1.7</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10058">HADOOP-10058</a>.
|
|
|
+ Minor bug reported by Akira AJISAKA and fixed by Chen He (metrics)<br>
|
|
|
+ <b>TestMetricsSystemImpl#testInitFirstVerifyStopInvokedImmediately fails on trunk</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10055">HADOOP-10055</a>.
|
|
|
+ Trivial bug reported by Eli Collins and fixed by Akira AJISAKA (documentation)<br>
|
|
|
+ <b>FileSystemShell.apt.vm doc has typo "numRepicas" </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10052">HADOOP-10052</a>.
|
|
|
+ Major sub-task reported by Andrew Wang and fixed by Andrew Wang (fs)<br>
|
|
|
+ <b>Temporarily disable client-side symlink resolution</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10047">HADOOP-10047</a>.
|
|
|
+ Major new feature reported by Gopal V and fixed by Gopal V (io)<br>
|
|
|
+ <b>Add a directbuffer Decompressor API to hadoop</b><br>
|
|
|
+ <blockquote>Direct Bytebuffer decompressors for Zlib (Deflate & Gzip) and Snappy </blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10046">HADOOP-10046</a>.
|
|
|
+ Trivial improvement reported by David S. Wang and fixed by David S. Wang <br>
|
|
|
+ <b>Print a log message when SSL is enabled</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10040">HADOOP-10040</a>.
|
|
|
+ Major bug reported by Yingda Chen and fixed by Chris Nauroth <br>
|
|
|
+ <b>hadoop.cmd in UNIX format and would not run by default on Windows</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10039">HADOOP-10039</a>.
|
|
|
+ Major bug reported by Suresh Srinivas and fixed by Haohui Mai (security)<br>
|
|
|
+ <b>Add Hive to the list of projects using AbstractDelegationTokenSecretManager</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10031">HADOOP-10031</a>.
|
|
|
+ Major bug reported by Chuan Liu and fixed by Chuan Liu (fs)<br>
|
|
|
+ <b>FsShell -get/copyToLocal/moveFromLocal should support Windows local path</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10030">HADOOP-10030</a>.
|
|
|
+ Major bug reported by Chuan Liu and fixed by Chuan Liu <br>
|
|
|
+ <b>FsShell -put/copyFromLocal should support Windows local path</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10029">HADOOP-10029</a>.
|
|
|
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas (fs)<br>
|
|
|
+ <b>Specifying har file to MR job fails in secure cluster</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10028">HADOOP-10028</a>.
|
|
|
+ Minor bug reported by Jing Zhao and fixed by Haohui Mai <br>
|
|
|
+ <b>Malformed ssl-server.xml.example </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10006">HADOOP-10006</a>.
|
|
|
+ Blocker bug reported by Junping Du and fixed by Junping Du (fs , util)<br>
|
|
|
+ <b>Compilation failure in trunk for o.a.h.fs.swift.util.JSONUtil</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-10005">HADOOP-10005</a>.
|
|
|
+ Trivial improvement reported by Jackie Chang and fixed by Jackie Chang <br>
|
|
|
+ <b>No need to check INFO severity level is enabled or not</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9998">HADOOP-9998</a>.
|
|
|
+ Major improvement reported by Junping Du and fixed by Junping Du (net)<br>
|
|
|
+ <b>Provide methods to clear only part of the DNSToSwitchMapping</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9982">HADOOP-9982</a>.
|
|
|
+ Major bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)<br>
|
|
|
+ <b>Fix dead links in hadoop site docs</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9981">HADOOP-9981</a>.
|
|
|
+ Critical bug reported by Kihwal Lee and fixed by Colin Patrick McCabe <br>
|
|
|
+ <b>globStatus should minimize its listStatus and getFileStatus calls</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9964">HADOOP-9964</a>.
|
|
|
+ Major bug reported by Junping Du and fixed by Junping Du (util)<br>
|
|
|
+ <b>O.A.H.U.ReflectionUtils.printThreadInfo() is not thread-safe which cause TestHttpServer pending 10 minutes or longer.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9956">HADOOP-9956</a>.
|
|
|
+ Major sub-task reported by Daryn Sharp and fixed by Daryn Sharp (ipc)<br>
|
|
|
+ <b>RPC listener inefficiently assigns connections to readers</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9955">HADOOP-9955</a>.
|
|
|
+ Major sub-task reported by Daryn Sharp and fixed by Daryn Sharp (ipc)<br>
|
|
|
+ <b>RPC idle connection closing is extremely inefficient</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9929">HADOOP-9929</a>.
|
|
|
+ Major bug reported by Jason Lowe and fixed by Colin Patrick McCabe (fs)<br>
|
|
|
+ <b>Insufficient permissions for a path reported as file not found</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9915">HADOOP-9915</a>.
|
|
|
+ Trivial improvement reported by Binglin Chang and fixed by Binglin Chang <br>
|
|
|
+ <b>o.a.h.fs.Stat support on Macosx</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9909">HADOOP-9909</a>.
|
|
|
+ Major improvement reported by Shinichi Yamashita and fixed by (fs)<br>
|
|
|
+ <b>org.apache.hadoop.fs.Stat should permit other LANG</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9908">HADOOP-9908</a>.
|
|
|
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon (util)<br>
|
|
|
+ <b>Fix NPE when versioninfo properties file is missing</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9898">HADOOP-9898</a>.
|
|
|
+ Minor bug reported by Todd Lipcon and fixed by Todd Lipcon (ipc , net)<br>
|
|
|
+ <b>Set SO_KEEPALIVE on all our sockets</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9897">HADOOP-9897</a>.
|
|
|
+ Trivial improvement reported by Binglin Chang and fixed by Binglin Chang (fs)<br>
|
|
|
+ <b>Add method to get path start position without drive specifier in o.a.h.fs.Path </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9889">HADOOP-9889</a>.
|
|
|
+ Major bug reported by Wei Yan and fixed by Wei Yan <br>
|
|
|
+ <b>Refresh the Krb5 configuration when creating a new kdc in Hadoop-MiniKDC</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9887">HADOOP-9887</a>.
|
|
|
+ Major bug reported by Chris Nauroth and fixed by Chuan Liu (fs)<br>
|
|
|
+ <b>globStatus does not correctly handle paths starting with a drive spec on Windows</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9875">HADOOP-9875</a>.
|
|
|
+ Minor bug reported by Aaron T. Myers and fixed by Aaron T. Myers (test)<br>
|
|
|
+ <b>TestDoAsEffectiveUser can fail on JDK 7</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9871">HADOOP-9871</a>.
|
|
|
+ Minor bug reported by Luke Lu and fixed by Junping Du <br>
|
|
|
+ <b>Fix intermittent findbug warnings in DefaultMetricsSystem</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9866">HADOOP-9866</a>.
|
|
|
+ Major test reported by Alejandro Abdelnur and fixed by Wei Yan (test)<br>
|
|
|
+ <b>convert hadoop-auth testcases requiring kerberos to use minikdc</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9865">HADOOP-9865</a>.
|
|
|
+ Major bug reported by Chuan Liu and fixed by Chuan Liu <br>
|
|
|
+ <b>FileContext.globStatus() has a regression with respect to relative path</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9860">HADOOP-9860</a>.
|
|
|
+ Major improvement reported by Wei Yan and fixed by Wei Yan <br>
|
|
|
+ <b>Remove class HackedKeytab and HackedKeytabEncoder from hadoop-minikdc once jira DIRSERVER-1882 solved</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9848">HADOOP-9848</a>.
|
|
|
+ Major new feature reported by Wei Yan and fixed by Wei Yan (security , test)<br>
|
|
|
+ <b>Create a MiniKDC for use with security testing</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9847">HADOOP-9847</a>.
|
|
|
+ Minor bug reported by Andrew Wang and fixed by Colin Patrick McCabe <br>
|
|
|
+ <b>TestGlobPath symlink tests fail to cleanup properly</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9833">HADOOP-9833</a>.
|
|
|
+ Minor improvement reported by Steve Loughran and fixed by Kousuke Saruta (build)<br>
|
|
|
+ <b>move slf4j to version 1.7.5</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9830">HADOOP-9830</a>.
|
|
|
+ Trivial bug reported by Dmitry Lysnichenko and fixed by Kousuke Saruta (documentation)<br>
|
|
|
+ <b>Typo at http://hadoop.apache.org/docs/current/</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9820">HADOOP-9820</a>.
|
|
|
+ Blocker bug reported by Daryn Sharp and fixed by Daryn Sharp (ipc , security)<br>
|
|
|
+ <b>RPCv9 wire protocol is insufficient to support multiplexing</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9817">HADOOP-9817</a>.
|
|
|
+ Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe <br>
|
|
|
+ <b>FileSystem#globStatus and FileContext#globStatus need to work with symlinks</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9806">HADOOP-9806</a>.
|
|
|
+ Major bug reported by Brandon Li and fixed by Brandon Li (nfs)<br>
|
|
|
+ <b>PortmapInterface should check if the procedure is out-of-range</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9791">HADOOP-9791</a>.
|
|
|
+ Major bug reported by Ivan Mitic and fixed by Ivan Mitic <br>
|
|
|
+ <b>Add a test case covering long paths for new FileUtil access check methods</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9787">HADOOP-9787</a>.
|
|
|
+ Major bug reported by Karthik Kambatla and fixed by Karthik Kambatla (util)<br>
|
|
|
+ <b>ShutdownHelper util to shutdown threads and threadpools</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9784">HADOOP-9784</a>.
|
|
|
+ Major improvement reported by Junping Du and fixed by Junping Du <br>
|
|
|
+ <b>Add a builder for HttpServer</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9748">HADOOP-9748</a>.
|
|
|
+ Critical sub-task reported by Daryn Sharp and fixed by Daryn Sharp (security)<br>
|
|
|
+ <b>Reduce blocking on UGI.ensureInitialized</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9703">HADOOP-9703</a>.
|
|
|
+ Minor bug reported by Mark Miller and fixed by Tsuyoshi OZAWA <br>
|
|
|
+ <b>org.apache.hadoop.ipc.Client leaks threads on stop.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9698">HADOOP-9698</a>.
|
|
|
+ Blocker sub-task reported by Daryn Sharp and fixed by Daryn Sharp (ipc)<br>
|
|
|
+ <b>RPCv9 client must honor server's SASL negotiate response</b><br>
|
|
|
+ <blockquote>The RPC client now waits for the Server's SASL negotiate response before instantiating its SASL client.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9693">HADOOP-9693</a>.
|
|
|
+ Trivial improvement reported by Steve Loughran and fixed by <br>
|
|
|
+ <b>Shell should add a probe for OSX</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9686">HADOOP-9686</a>.
|
|
|
+ Major improvement reported by Jason Lowe and fixed by Jason Lowe (conf)<br>
|
|
|
+ <b>Easy access to final parameters in Configuration</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9683">HADOOP-9683</a>.
|
|
|
+ Blocker sub-task reported by Luke Lu and fixed by Daryn Sharp (ipc)<br>
|
|
|
+ <b>Wrap IpcConnectionContext in RPC headers</b><br>
|
|
|
+ <blockquote>Connection context is now sent as a rpc header wrapped protobuf.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9660">HADOOP-9660</a>.
|
|
|
+ Major bug reported by Enis Soztutar and fixed by Enis Soztutar (scripts , util)<br>
|
|
|
+ <b>[WINDOWS] Powershell / cmd parses -Dkey=value from command line as [-Dkey, value] which breaks GenericsOptionParser</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9652">HADOOP-9652</a>.
|
|
|
+ Major improvement reported by Colin Patrick McCabe and fixed by Andrew Wang <br>
|
|
|
+ <b>Allow RawLocalFs#getFileLinkStatus to fill in the link owner and mode if requested</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9635">HADOOP-9635</a>.
|
|
|
+ Major bug reported by V. Karthik Kumar and fixed by (native)<br>
|
|
|
+ <b>Fix Potential Stack Overflow in DomainSocket.c</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9623">HADOOP-9623</a>.
|
|
|
+ Major improvement reported by Timothy St. Clair and fixed by Amandeep Khurana (fs/s3)<br>
|
|
|
+ <b>Update jets3t dependency to 0.9.0 </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9618">HADOOP-9618</a>.
|
|
|
+ Major new feature reported by Todd Lipcon and fixed by Todd Lipcon (util)<br>
|
|
|
+ <b>Add thread which detects JVM pauses</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9611">HADOOP-9611</a>.
|
|
|
+ Major improvement reported by Timothy St. Clair and fixed by Timothy St. Clair (build)<br>
|
|
|
+ <b>mvn-rpmbuild against google-guice > 3.0 yields missing cglib dependency</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9598">HADOOP-9598</a>.
|
|
|
+ Major test reported by Aleksey Gorshkov and fixed by Andrey Klochkov <br>
|
|
|
+ <b>Improve code coverage of RMAdminCLI</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9594">HADOOP-9594</a>.
|
|
|
+ Major improvement reported by Timothy St. Clair and fixed by Timothy St. Clair (build)<br>
|
|
|
+ <b>Update apache commons math dependency</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9582">HADOOP-9582</a>.
|
|
|
+ Major bug reported by Ashwin Shankar and fixed by Ashwin Shankar (conf)<br>
|
|
|
+ <b>Non-existent file to "hadoop fs -conf" doesn't throw error</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9527">HADOOP-9527</a>.
|
|
|
+ Major bug reported by Arpit Agarwal and fixed by Arpit Agarwal (fs , test)<br>
|
|
|
+ <b>Add symlink support to LocalFileSystem on Windows</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9515">HADOOP-9515</a>.
|
|
|
+ Major new feature reported by Brandon Li and fixed by Brandon Li <br>
|
|
|
+ <b>Add general interface for NFS and Mount</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9509">HADOOP-9509</a>.
|
|
|
+ Major new feature reported by Brandon Li and fixed by Brandon Li <br>
|
|
|
+ <b>Implement ONCRPC and XDR</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9494">HADOOP-9494</a>.
|
|
|
+ Major improvement reported by Dennis Y and fixed by Andrey Klochkov <br>
|
|
|
+ <b>Excluded auto-generated and examples code from clover reports</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9487">HADOOP-9487</a>.
|
|
|
+ Major improvement reported by Steve Loughran and fixed by (conf)<br>
|
|
|
+ <b>Deprecation warnings in Configuration should go to their own log or otherwise be suppressible</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9470">HADOOP-9470</a>.
|
|
|
+ Major improvement reported by Ivan A. Veselovsky and fixed by Ivan A. Veselovsky (test)<br>
|
|
|
+ <b>eliminate duplicate FQN tests in different Hadoop modules</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9432">HADOOP-9432</a>.
|
|
|
+ Minor new feature reported by Steve Loughran and fixed by (build , documentation)<br>
|
|
|
+ <b>Add support for markdown .md files in site documentation</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9421">HADOOP-9421</a>.
|
|
|
+ Blocker sub-task reported by Sanjay Radia and fixed by Daryn Sharp <br>
|
|
|
+ <b>Convert SASL to use ProtoBuf and provide negotiation capabilities</b><br>
|
|
|
+ <blockquote>Raw SASL protocol now uses protobufs wrapped with RPC headers.
|
|
|
+The negotiation sequence incorporates the state of the exchange.
|
|
|
+The server now has the ability to advertise its supported auth types.</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9420">HADOOP-9420</a>.
|
|
|
+ Major bug reported by Todd Lipcon and fixed by Liang Xie (ipc , metrics)<br>
|
|
|
+ <b>Add percentile or max metric for rpcQueueTime, processing time</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9417">HADOOP-9417</a>.
|
|
|
+ Major sub-task reported by Andrew Wang and fixed by Andrew Wang (fs)<br>
|
|
|
+ <b>Support for symlink resolution in LocalFileSystem / RawLocalFileSystem</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9350">HADOOP-9350</a>.
|
|
|
+ Minor bug reported by Steve Loughran and fixed by Robert Kanter (build)<br>
|
|
|
+ <b>Hadoop not building against Java7 on OSX </b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9319">HADOOP-9319</a>.
|
|
|
+ Major improvement reported by Arpit Agarwal and fixed by Binglin Chang <br>
|
|
|
+ <b>Update bundled lz4 source to latest version</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9291">HADOOP-9291</a>.
|
|
|
+ Major test reported by Ivan A. Veselovsky and fixed by Ivan A. Veselovsky <br>
|
|
|
+ <b>enhance unit-test coverage of package o.a.h.metrics2</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9254">HADOOP-9254</a>.
|
|
|
+ Major test reported by Vadim Bondarev and fixed by Vadim Bondarev <br>
|
|
|
+ <b>Cover packages org.apache.hadoop.util.bloom, org.apache.hadoop.util.hash</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9241">HADOOP-9241</a>.
|
|
|
+ Trivial improvement reported by Harsh J and fixed by Harsh J <br>
|
|
|
+ <b>DU refresh interval is not configurable</b><br>
|
|
|
+ <blockquote>The 'du' (disk usage command from Unix) script refresh monitor is now configurable in the same way as its 'df' counterpart, via the property 'fs.du.interval', the default of which is 10 minute (in ms).</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9225">HADOOP-9225</a>.
|
|
|
+ Major test reported by Vadim Bondarev and fixed by Andrey Klochkov <br>
|
|
|
+ <b>Cover package org.apache.hadoop.compress.Snappy</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9199">HADOOP-9199</a>.
|
|
|
+ Major test reported by Vadim Bondarev and fixed by Andrey Klochkov <br>
|
|
|
+ <b>Cover package org.apache.hadoop.io with unit tests</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9114">HADOOP-9114</a>.
|
|
|
+ Minor bug reported by liuyang and fixed by sathish <br>
|
|
|
+ <b>After defined the dfs.checksum.type as the NULL, write file and hflush will through java.lang.ArrayIndexOutOfBoundsException</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9078">HADOOP-9078</a>.
|
|
|
+ Major test reported by Ivan A. Veselovsky and fixed by Ivan A. Veselovsky <br>
|
|
|
+ <b>enhance unit-test coverage of class org.apache.hadoop.fs.FileContext</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9063">HADOOP-9063</a>.
|
|
|
+ Minor test reported by Ivan A. Veselovsky and fixed by Ivan A. Veselovsky <br>
|
|
|
+ <b>enhance unit-test coverage of class org.apache.hadoop.fs.FileUtil</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-9016">HADOOP-9016</a>.
|
|
|
+ Minor bug reported by Ivan A. Veselovsky and fixed by Ivan A. Veselovsky <br>
|
|
|
+ <b>org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream.HarFsInputStream.skip(long) must never return negative value.</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8814">HADOOP-8814</a>.
|
|
|
+ Minor improvement reported by Brandon Li and fixed by Brandon Li (conf , fs , fs/s3 , ha , io , metrics , performance , record , security , util)<br>
|
|
|
+ <b>Inefficient comparison with the empty string. Use isEmpty() instead</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8753">HADOOP-8753</a>.
|
|
|
+ Minor bug reported by Nishan Shetty, Huawei and fixed by Benoy Antony <br>
|
|
|
+ <b>LocalDirAllocator throws "ArithmeticException: / by zero" when there is no available space on configured local dir</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8704">HADOOP-8704</a>.
|
|
|
+ Major improvement reported by Thomas Graves and fixed by Jonathan Eagles <br>
|
|
|
+ <b>add request logging to jetty/httpserver</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8545">HADOOP-8545</a>.
|
|
|
+ Major new feature reported by Tim Miller and fixed by Dmitry Mezhensky (fs)<br>
|
|
|
+ <b>Filesystem Implementation for OpenStack Swift</b><br>
|
|
|
+ <blockquote>Added file system implementation for OpenStack Swift.
|
|
|
+There are two implementation: block and native (similar to Amazon S3 integration).
|
|
|
+Data locality issue solved by patch in Swift, commit procedure to OpenStack is in progress.
|
|
|
+
|
|
|
+To use implementation add to core-site.xml following:
|
|
|
+...
|
|
|
+ <property>
|
|
|
+ <name>fs.swift.impl</name>
|
|
|
+ <value>com.mirantis.fs.SwiftFileSystem</value>
|
|
|
+ </property>
|
|
|
+ <property>
|
|
|
+ <name>fs.swift.block.impl</name>
|
|
|
+ <value>com.mirantis.fs.block.SwiftBlockFileSystem</value>
|
|
|
+ </property>
|
|
|
+...
|
|
|
+
|
|
|
+In MapReduce job specify following configs for OpenStack Keystone authentication:
|
|
|
+conf.set("swift.auth.url", "http://172.18.66.117:5000/v2.0/tokens");
|
|
|
+conf.set("swift.tenant", "superuser");
|
|
|
+conf.set("swift.username", "admin1");
|
|
|
+conf.set("swift.password", "password");
|
|
|
+conf.setInt("swift.http.port", 8080);
|
|
|
+conf.setInt("swift.https.port", 443);
|
|
|
+
|
|
|
+Additional information specified on github: https://github.com/DmitryMezhensky/Hadoop-and-Swift-integration</blockquote></li>
|
|
|
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7344">HADOOP-7344</a>.
|
|
|
+ Major bug reported by Daryn Sharp and fixed by Colin Patrick McCabe (fs)<br>
|
|
|
+ <b>globStatus doesn't grok groupings with a slash</b><br>
|
|
|
+ <blockquote></blockquote></li>
|
|
|
+</ul>
|
|
|
+</body></html>
|
|
|
<!--
|
|
|
Licensed to the Apache Software Foundation (ASF) under one or more
|
|
|
contributor license agreements. See the NOTICE file distributed with
|
Jing Zhao