CHANGES.txt 86 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036103710381039104010411042104310441045104610471048104910501051105210531054105510561057105810591060106110621063106410651066106710681069107010711072107310741075107610771078107910801081108210831084108510861087108810891090109110921093109410951096109710981099110011011102110311041105110611071108110911101111111211131114111511161117111811191120112111221123112411251126112711281129113011311132113311341135113611371138113911401141114211431144114511461147114811491150115111521153115411551156115711581159116011611162116311641165116611671168116911701171117211731174117511761177117811791180118111821183118411851186118711881189119011911192119311941195119611971198119912001201120212031204120512061207120812091210121112121213121412151216121712181219122012211222122312241225122612271228122912301231123212331234123512361237123812391240124112421243124412451246124712481249125012511252125312541255125612571258125912601261126212631264126512661267126812691270127112721273127412751276127712781279128012811282128312841285128612871288128912901291129212931294129512961297129812991300130113021303130413051306130713081309131013111312131313141315131613171318131913201321132213231324132513261327132813291330133113321333133413351336133713381339134013411342134313441345134613471348134913501351135213531354135513561357135813591360136113621363136413651366136713681369137013711372137313741375137613771378137913801381138213831384138513861387138813891390139113921393139413951396139713981399140014011402140314041405140614071408140914101411141214131414141514161417141814191420142114221423142414251426142714281429143014311432143314341435143614371438143914401441144214431444144514461447144814491450145114521453145414551456145714581459146014611462146314641465146614671468146914701471147214731474147514761477147814791480148114821483148414851486148714881489149014911492149314941495149614971498149915001501150215031504150515061507150815091510151115121513151415151516151715181519152015211522152315241525152615271528152915301531153215331534153515361537153815391540154115421543154415451546154715481549155015511552155315541555155615571558155915601561156215631564156515661567156815691570157115721573157415751576157715781579158015811582158315841585158615871588158915901591159215931594159515961597159815991600160116021603160416051606160716081609161016111612161316141615161616171618161916201621162216231624162516261627162816291630163116321633163416351636163716381639164016411642164316441645164616471648164916501651165216531654165516561657165816591660166116621663166416651666166716681669167016711672167316741675167616771678167916801681168216831684168516861687168816891690169116921693169416951696169716981699170017011702170317041705170617071708170917101711171217131714171517161717171817191720172117221723172417251726172717281729173017311732173317341735173617371738173917401741174217431744174517461747174817491750175117521753175417551756175717581759176017611762176317641765176617671768176917701771177217731774177517761777177817791780178117821783178417851786178717881789179017911792179317941795179617971798179918001801180218031804180518061807180818091810181118121813181418151816181718181819182018211822182318241825182618271828182918301831183218331834183518361837183818391840184118421843184418451846184718481849185018511852185318541855185618571858185918601861186218631864186518661867186818691870187118721873187418751876187718781879188018811882188318841885188618871888188918901891189218931894189518961897189818991900190119021903190419051906190719081909191019111912191319141915191619171918191919201921192219231924192519261927192819291930193119321933193419351936193719381939194019411942194319441945194619471948194919501951195219531954195519561957195819591960196119621963196419651966196719681969197019711972197319741975197619771978197919801981198219831984198519861987198819891990199119921993199419951996199719981999200020012002200320042005200620072008200920102011201220132014201520162017201820192020202120222023202420252026202720282029203020312032203320342035203620372038203920402041204220432044204520462047204820492050205120522053205420552056205720582059206020612062206320642065206620672068206920702071207220732074207520762077207820792080208120822083208420852086208720882089209020912092209320942095209620972098209921002101210221032104210521062107210821092110211121122113211421152116211721182119212021212122212321242125212621272128212921302131213221332134213521362137213821392140214121422143214421452146214721482149215021512152215321542155215621572158215921602161216221632164216521662167216821692170217121722173217421752176217721782179218021812182218321842185218621872188218921902191219221932194219521962197219821992200220122022203220422052206220722082209221022112212221322142215221622172218221922202221222222232224222522262227222822292230223122322233223422352236223722382239224022412242224322442245224622472248224922502251225222532254225522562257225822592260226122622263226422652266226722682269227022712272227322742275227622772278227922802281228222832284228522862287228822892290229122922293
  1. Hadoop MapReduce Change Log
  2. Release 0.22.1 - Unreleased
  3. INCOMPATIBLE CHANGES
  4. NEW FEATURES
  5. IMPROVEMENTS
  6. MAPREDUCE-4403. Test case for job resubmission in TestRecoveryManager.
  7. (Mayank Bansal via shv)
  8. MAPREDUCE-4404. Test case for TestMRJobClient to verify user name.
  9. (Mayank Bansal via shv)
  10. MAPREDUCE-4405. Test case for HierarchicalQueue in TestJobQueueClient.
  11. (Mayank Bansal via shv)
  12. MAPREDUCE-4349. Add test to verify Distributed Cache consistency when
  13. cached archives are deleted. (Mayank Bansal via shv)
  14. OPTIMIZATIONS
  15. BUG FIXES
  16. MAPREDUCE-3593. Fix user impersonation. (Mayank Bansal via shv)
  17. MAPREDUCE-2539. Fixed NPE in getMapTaskReports in JobClient.
  18. (Robert Evans via shv)
  19. MAPREDUCE-3725. hadoop job -list returns user name as NULL.
  20. (Mayank Bansal via shv)
  21. MAPREDUCE-3837. Job tracker is not able to recover jobs after crash.
  22. (Mayank Bansal via shv)
  23. MAPREDUCE-4164. Fix Communication exception thrown after task completion.
  24. (Mayank Bansal via shv)
  25. MAPREDUCE-3863. 0.22 branch mvn deploy is not publishing hadoop-streaming JAR
  26. (Benoy Antony via cos)
  27. SUBTASKS OF HADOOP-8357. Restore security in Hadoop 0.22 branch.
  28. MAPREDUCE-4240. Revert MAPREDUCE-2767 (Benoy Antony via shv)
  29. MAPREDUCE-4243. Modify mapreduce build to include task-controller target.
  30. (Benoy Antony via shv)
  31. MAPREDUCE-2178. Race condition in LinuxTaskController permissions handling.
  32. (Todd Lipcon, Benoy Antony via shv)
  33. MAPREDUCE-4244. Set supplemental groups correctly. (Benoy Antony via shv)
  34. MAPREDUCE-2420. JobTracker should be able to renew delegation token
  35. over HTTP. (Boris Shkolnik and Benoy Antony via shv)
  36. MAPREDUCE-2452. Moves the cancellation of delegation tokens to a separate
  37. thread. (Devaraj Das and Benoy Antony via shv)
  38. MAPREDUCE-2651. Fix race condition in Linux task controller for
  39. job log directory creation. (Bharath Mundlapudi and Benoy Antony via shv)
  40. MAPREDUCE-2353. MR changes to reflect the API changes in SecureIO library.
  41. (Devaraj Das and Benoy Antony via shv)
  42. MAPREDUCE-2377. task-controller fails to parse configuration if it doesn't
  43. end in \n. (Todd Lipcon and Benoy Antony via shv)
  44. MAPREDUCE-2103. Additional changes to task-controller.c
  45. (Benoy Antony via shv)
  46. MAPREDUCE-2376. test-task-controller fails if run as a userid < 1000.
  47. (Todd Lipcon and Benoy Antony via shv)
  48. MAPREDUCE-4314. Synchronization in JvmManager. (Benoy Antony via shv)
  49. MAPREDUCE-4246. Failure in deleting user directories in secure Hadoop.
  50. (Benoy Antony via shv)
  51. MAPREDUCE-4247. Fix TestTaskTrackerLocalization. (Benoy Antony via shv)
  52. MAPREDUCE-4248. Fix TestRecoveryManager. (Benoy Antony via shv)
  53. MAPREDUCE-4248. Fix TestRecoveryManager. (Benoy Antony via shv)
  54. MAPREDUCE-4249. Fix streaming test TestFileArgs. (Benoy Antony via shv)
  55. MAPREDUCE-4318. TestRecoveryManager should not use raw configuration keys.
  56. (Benoy Antony via shv)
  57. END OF HADOOP-8357 SUBTASKS
  58. MAPREDUCE-4342. Distributed Cache gives inconsistent result if cache files
  59. get deleted from task tracker. (Mayank Bansal via shv)
  60. MAPREDUCE-4360. Capacity Scheduler Hierarchical leaf queue does not honor
  61. the max capacity of container queue. (Mayank Bansal via shv)
  62. MAPREDUCE-5706. toBeDeleted parent directories aren't being cleaned up.
  63. (Robert Kanter via kasha)
  64. Release 0.22.0 - 2011-11-29
  65. INCOMPATIBLE CHANGES
  66. MAPREDUCE-1866. Removes deprecated class
  67. org.apache.hadoop.streaming.UTF8ByteArrayUtils. (amareshwari)
  68. MAPREDUCE-1664. Changes the behaviour of the combination of job-acls
  69. when they function together with queue-acls. (Ravi Gummadi via vinodkv)
  70. NEW FEATURES
  71. MAPREDUCE-1804. Stress-test tool for HDFS introduced in HDFS-708.
  72. (Joshua Harlow via shv)
  73. MAPREDUCE-220. Collect cpu and memory statistics per task. (Scott Chen via
  74. acmurthy)
  75. MAPREDUCE-1970. Reed-Solomon code implementation for HDFS RAID.
  76. (Scott Chen via dhruba)
  77. MAPREDUCE-2169. Integrated Reed-Solomon code with RaidNode. (Ramkumar
  78. Vadali via schen)
  79. IMPROVEMENTS
  80. MAPREDUCE-2141. Add an "extra data" field to Task for use by Mesos. (matei)
  81. MAPREDUCE-2140. Regenerate fair scheduler design doc PDF. (matei)
  82. MAPREDUCE-1546. Redirect all job pages to corresponding history page
  83. if job is not in JT memory. (Scott Chen via sharad)
  84. MAPREDUCE-1092. Enable assertions for unit tests. (Eli Collins via
  85. cdouglas)
  86. MAPREDUCE-1680. Add a metric recording JobTracker heartbeats processed.
  87. (Dick King via cdouglas)
  88. MAPREDUCE-1761. FairScheduler allows separate configuration of node
  89. and rack locality wait time (Scott Chen via dhruba)
  90. MAPREDUCE-1539. authorization checks for inter-server protocol
  91. (based on HADOOP-6600) (Boris Shkolnik via shv)
  92. MAPREDUCE-1798. Names the configuration keys for the Kerberos
  93. principals better. (Boris Shkolnik via ddas)
  94. MAPREDUCE-1773. streaming doesn't support jobclient.output.filter.
  95. (Amareshwari Sriramadasu via vinodkv)
  96. MAPREDUCE-1785. Add streaming config option for not emitting the key.
  97. (Eli Collins via sharad)
  98. MAPREDUCE-572. If #link is missing from uri format of -cacheArchive
  99. then streaming does not throw error. (Amareshwari Sriramadasu via
  100. vinodkv)
  101. MAPREDUCE-1545. Add timestamps for first task type launched in job summary.
  102. (Luke Lu via cdouglas)
  103. MAPREDUCE-1543. Add an audit log for authentication events. (Amar Kamat and
  104. Luke Lu via cdouglas)
  105. MAPREDUCE-1762. Add ability to set values of task counters. (Scott Chen via
  106. cdouglas)
  107. MAPREDUCE-1533. Reduce overhead of logging and string manipulation during
  108. heartbeat processing. (Amar Kamat and Dick King via cdouglas)
  109. MAPREDUCE-1516. JobTracker issues delegation tokens only if the user's
  110. authentication is Kerberos. (Jitendra Pandey via ddas)
  111. MAPREDUCE-647. Update distcp forrest documentation to reflect the changes
  112. of HADOOP-5472, MAPREDUCE-642 and HADOOP-5620. (Rodrigo Schmidt via
  113. szetszwo)
  114. MAPREDUCE-1851. Documents configuration parameters in streaming.
  115. (amareshwari)
  116. MAPREDUCE-1868. Add a read and connection timeout to JobClient while
  117. pulling tasklogs. (Krishna Ramachandran via acmurthy)
  118. MAPREDUCE-1778. Ensure failure to setup CompletedJobStatusStore is not
  119. silently ignored by the JobTracker. (Krishna Ramachandran via acmurthy)
  120. MAPREDUCE-1850. Includes job submit host information (name and ip) in
  121. jobconf and jobdetails display (Krishna Ramachandran via amareshwari)
  122. MAPREDUCE-1893. Slive with multiple reducers. (shv)
  123. MAPREDUCE-1248. Fixes redudant memory copying in StreamKeyValUtil.
  124. (Ruibang He via amareshwari)
  125. MAPREDUCE-1840. Enhancements to Gridmix benchmark simulating user
  126. diversity, queue replay, and task duration for JobTracker load testing.
  127. Also includes compatibility with security enhancements, and scalability
  128. improvements. (Amar Kamat, Rahul Singh, Hong Tang, and cdouglas)
  129. MAPREDUCE-1848. Put number of speculative, data local, rack local
  130. tasks in JobTracker metrics. (Scott Chen via dhruba)
  131. MAPREDUCE-1935. Makes the Distcp to work in a secure environment.
  132. (Boris Shkolnik via ddas)
  133. MAPREDUCE-1945. The MapReduce component for HADOOP-6632.
  134. (Kan Zhang & Jitendra Pandey via ddas)
  135. MAPREDUCE-1936. Modify Gridmix3 to support more tunable parameters for
  136. stress submission and sleep jobs. (Hong Tang via cdouglas)
  137. MAPREDUCE-1733. Makes pipes applications secure. (Jitendra Pandey via ddas)
  138. MAPREDUCE-1566. Adds a configuration attribute using which job clients can
  139. specify a credentials file. The tokens from there will be passed to the job.
  140. (Jitendra Pandey and Owen O'Malley via ddas)
  141. MAPREDUCE-1624. Documents the job credentials and associated details to do
  142. with delegation tokens (on the client side).
  143. (Jitendra Pandey and Devaraj Das via ddas)
  144. MAPREDUCE-1834. TestSimulatorDeterministicReplay timesout on trunk.
  145. (Hong Tang via mahadev)
  146. MAPREDUCE-1993. Fixes test failure
  147. TestTrackerDistributedCacheManagerWithLinuxTaskController. (Devaraj Das
  148. via amareshwari)
  149. MAPREDUCE-1523. Making Mumak work with Capacity-Scheduler (Anirban Das
  150. via mahadev)
  151. MAPREDUCE-1920. Enables completed jobstatus store by default. (Tom White
  152. via amareshwari)
  153. MAPREDUCE-1881. Improve TaskTrackerInstrumentation to enable collection of
  154. advanced metrics. (Matei Zaharia via acmurthy)
  155. MAPREDUCE-1548. Hadoop archives preserve times and other properties from
  156. original files. (Rodrigo Schmidt via dhruba)
  157. MAPREDUCE-1517. Supports streaming job to run in the background. (Bochun Bai
  158. via amareshwari)
  159. MAPREDUCE-1819. RaidNode is now smarter in submitting Raid jobs. (Ramkumar
  160. Vadali via schen)
  161. MAPREDUCE-2132. A command line option in RaidShell to fix blocks using raid
  162. MAPREDUCE-2147. Remove redundant lines in JobInProgress ctor.
  163. (Harsh J Chouraria via cutting)
  164. HADOOP-7007. Update the hudson-test-patch ant target to work with the
  165. latest test-patch.sh script (gkesavan)
  166. MAPREDUCE-1818. RaidNode can specify scheduling parameters. (Ramkumar
  167. Vadali via schen)
  168. MAPREDUCE-2051. Contribute a fair scheduler preemption system test.
  169. (Todd Lipcon via tomwhite)
  170. MAPREDUCE-1892. RaidNode can allow layered policies more efficiently.
  171. (Ramkumar Vadali via schen)
  172. MAPREDUCE-1592. Generate Eclipse's .classpath file from Ivy config.
  173. (tomwhite via nigel)
  174. MAPREDUCE-2073. TestTrackerDistributedCacheManager should be up-front
  175. about requirements on build environment. (Todd Lipcon via tomwhite)
  176. MAPREDUCE-2093. Herriot JT and TT clients should vend statistics. (cos)
  177. MAPREDUCE-2167. Faster directory traversal for raid node. (Ramkumar Vadali
  178. via schen)
  179. MAPREDUCE-1931. Gridmix forrest documentation . (Ranjit Mathew via vinodkv).
  180. MAPREDUCE-1878. Add MRUnit documentation. (Aaron Kimball via tomwhite)
  181. MAPREDUCE-2180. Add coverage of fair scheduler servlet to system test (todd)
  182. MAPREDUCE-2260. Remove auto-generated native build files. (rvs via eli)
  183. MAPREDUCE-2314. configure files that are generated as part of the released
  184. tarball need to have executable bit set (rvs via cos)
  185. MAPREDUCE-1159. Limit Job name in web UI to be 80 char long. (Harsh J
  186. Chouraria via szetszwo)
  187. MAPREDUCE-2337. Remove dependence of public MapReduce API on classes in
  188. server package. (tomwhite)
  189. MAPREDUCE-869. Documentation for config to set map/reduce task environment
  190. (Alejandro Abdelnur via todd)
  191. MAPREDUCE-2410. Add entry to streaming FAQ about how streaming reducers
  192. receive keys. (Harsh J Chouraria via todd)
  193. MAPREDUCE-2383. Improve documentation of DistributedCache methods (Harsh J
  194. Chouraria via todd)
  195. MAPREDUCE-2222. Ivy resolve force mode should be turned off by default.
  196. (Luke Lu via tomwhite)
  197. MAPREDUCE-2103. task-controller shouldn't require o-r permissions.
  198. (todd via eli)
  199. MAPREDUCE-2499. MR part of HADOOP-7291. (eli)
  200. MAPREDUCE-2505. Explain how to use ACLs in the fair scheduler.
  201. (matei via eli)
  202. MAPREDUCE-2502. JobSubmitter should use mapreduce.job.maps instead of
  203. its deprecated equivalent. (eli via todd)
  204. MAPREDUCE-2372. TaskLogAppender mechanism shouldn't be set up in
  205. log4j.properties (todd)
  206. MAPREDUCE-2516. Rename webinterface.private.actions to
  207. mapreduce.jobtracker.webinterface.trusted (Ari Rabkin via todd)
  208. HADOOP-7106. Reorganize project SVN layout to "unsplit" the projects.
  209. (todd, nigel)
  210. MAPREDUCE-3039. Upgrade to Avro 1.5.3. (Joep Rottinghuis via shv)
  211. MAPREDUCE-3138. Add a utility to help applications bridge changes in
  212. Context Objects APIs due to MAPREDUCE-954. (omalley via acmurthy)
  213. OPTIMIZATIONS
  214. MAPREDUCE-1354. Enhancements to JobTracker for better performance and
  215. scalability. (Arun C. Murthy & Richard King via acmurthy)
  216. MAPREDUCE-1829. JobInProgress.findSpeculativeTask should use min() to
  217. find the candidate instead of sort(). (Scott Chen via vinodkv)
  218. MAPREDUCE-2026. Make JobTracker.getJobCounters() and
  219. JobInProgress.getCounters() aquire locks in a shorter time period.
  220. (Joydeep Sen Sarma via schen)
  221. BUG FIXES
  222. MAPREDUCE-1845. FairScheduler.tasksToPreempt() can return negative number.
  223. (Scott Chen via matei)
  224. MAPREDUCE-1707. TaskRunner can get NPE in getting ugi from TaskTracker.
  225. (Vinod Kumar Vavilapalli)
  226. MAPREDUCE-1532. Ensures that delegation tokens is obtained as the
  227. actual user when the proxy-user is used for submitting jobs. Also
  228. refactors the DelegationTokenToRenew class. (ddas)
  229. MAPREDUCE-1558. Fixes MRAdmin to look up the principal of the
  230. JobTracker and use that in the RefreshUserToGroupsMapping protocol and
  231. RefreshAuthorizationPolicyProtocol. (Boris Shkolnik via ddas)
  232. MAPREDUCE-1662. Remove unused methods from TaskRunner. (Amareshwari
  233. Sriramadasu via cdouglas)
  234. MAPREDUCE-1617. Use IPv4 stack for unit tests. (Amar Kamat and Luke Lu via
  235. cdouglas)
  236. MAPREDUCE-1599. Fixes MRBench so that it reuses tokens across jobs
  237. correctly. (Jitendra Nath Pandey via ddas)
  238. MAPREDUCE-1836. Refresh for proxy superuser config (mr part for HDFS-1096).
  239. (Boris Shkolnik via shv)
  240. MAPREDUCE-1505. Create RPC client on job submission, not in cstr of Job
  241. instance. (Dick King via cdouglas)
  242. MAPREDUCE-1813. NPE in PipeMapred.MRErrorThread. (Ravi Gummadi via vinodkv)
  243. MAPREDUCE-1225. Fixes DistributedCache to check if the file is fresh or not,
  244. for the first localization also. (Zhong Wang via amareshwari)
  245. MAPREDUCE-1559. Fixes the token renewer to use the JobTracker's
  246. credentials for talking to the NameNode. (ddas)
  247. MAPREDUCE-1492. Delete obsolete har files used on the parity files
  248. of hdfs raid. (Rodrigo Schmidt via dhruba)
  249. MAPREDUCE-1857. Removes unused configuration parameters in streaming.
  250. (amareshwari)
  251. MAPREDUCE-1887. MRAsyncDiskService now properly absolutizes volume root
  252. paths. (Aaron Kimball via zshao)
  253. MAPREDUCE-1863. Fix NPE in Rumen when processing null CDF for failed task
  254. attempts. (Amar Kamat via cdouglas)
  255. MAPREDUCE-1864. Removes uninitialized/unused variables in
  256. org.apache.hadoop.streaming.PipeMapRed. (amareshwari)
  257. MAPREDUCE-1888. Fixes Streaming to override output key and value types,
  258. only if mapper/reducer is a command. (Ravi Gummadi via amareshwari)
  259. MAPREDUCE-577. Fixes duplicate records in StreamXmlRecordReader.
  260. (Ravi Gummadi via amareshwari)
  261. MAPREDUCE-1894. Fixed a bug in DistributedRaidFileSystem.readFully()
  262. that was causing it to loop infinitely. (Ramkumar Vadali via dhruba)
  263. MAPREDUCE-1838. Reduce the time needed for raiding a bunch of files
  264. by randomly assigning files to map tasks. (Ramkumar Vadali via dhruba)
  265. MAPREDUCE-1820. Fix InputSampler to clone sampled keys. (Alex Kozlov via
  266. cdouglas)
  267. MAPREDUCE-1528. Incorporates the changes to the credentials API done in
  268. HADOOP-6845. Also, introduces Credentials in JobConf, and in JobContext.
  269. (Jitendra Pandey and Arun Murthy via ddas)
  270. MAPREDUCE-1865. Rumen should also support jobhistory files generated using
  271. trunk. (Amar Kamat via amareshwari)
  272. MAPREDUCE-1621. Fixes NPE in TextOutputReader.getLastOutput if it has never
  273. read any output. (amareshwari)
  274. MAPREDUCE-1911. Fixes errors in -info message in streaming. (amareshwari)
  275. MAPREDUCE-1772. Corrects errors in streaming documentation in forrest.
  276. (amareshwari)
  277. MAPREDUCE-1925. Fix failing TestRumenJobTraces. (Ravi Gummadi via cdouglas)
  278. MAPREDUCE-1718. Fixes a bug in the construction of jobconf key for the
  279. mapping that the tasks use at runtime for looking up delegation tokens.
  280. (Boris Shkolnik via ddas)
  281. MAPREDUCE-1701. Fixes a problem to do with exception handling in
  282. delegation-token renewals. (Boris Shkolnik via ddas)
  283. MAPREDUCE-1686. Fixes StreamUtil.goodClassOrNull to find classes without
  284. package names. (Paul Burkhardt via amareshwari)
  285. MAPREDUCE-1288. Fixes TrackerDistributedCacheManager to take into account
  286. the owner of the localized file in the mapping from cache URIs to
  287. CacheStatus objects. (ddas)
  288. MAPREDUCE-1982. Fixes Rumen's TraceBuilder to extract job name from either
  289. of configuration properties "mapreduce.job.name" and "mapred.job.name".
  290. (Ravi Gummadi via amareshwari)
  291. MAPREDUCE-1958. The MapReduce part corresponding to the HADOOP-6873.
  292. (Boris Shkolnik & Owen O'Malley via ddas)
  293. MAPREDUCE-1900. TaskTracker and JobTracker closes FileSystems, opened on
  294. behalf of users that it no longer requires. (Kan Zhang and ddas via ddas)
  295. MAPREDUCE-1992. Fixes a problem to do with bringing up the JobTracker in
  296. unsecure mode. (Kan Zhang via ddas)
  297. MAPREDUCE-1999. Fixes ClientProtocol to use the correct
  298. DelegationTokenSelector. (Jitendra Pandey via ddas)
  299. MAPREDUCE-1780. AccessControlList.toString() is used for serialization of
  300. ACL in JobStatus.java. (Ravi Gummadi via vinodkv)
  301. MAPREDUCE-1961. Fix ConcurrentModificationException in Gridmix during
  302. shutdown. (Hong Tang via cdouglas)
  303. MAPREDUCE-2000. Fix parsing of JobHistory lines in Rumen when quotes are
  304. escaped. (Hong Tang via cdouglas)
  305. MAPREDUCE-2022. Fixes compilation errors in TestSubmitJob. (amareshwari)
  306. MAPREDUCE-1670. RAID policies should not scan their own destination path.
  307. (Ramkumar Vadali via dhruba)
  308. MAPREDUCE-1668. RaidNode Hars a directory only if all its parity files
  309. have been created. (Ramkumar Vadali via dhruba)
  310. MAPREDUCE-2021. Fixes duplicate hostnames in CombineFileInputFormat's
  311. split locations. (amareshwari)
  312. MAPREDUCE-1375. Fixes flaky test TestFileArgs. (Todd Lipcon via
  313. amareshwari)
  314. MAPREDUCE-2023. TestDFSIO should not stop reading if curSize != bufferSize.
  315. (Hong Tang via szetszwo)
  316. MAPREDUCE-2031. Fixes test failures TestTaskLauncher and
  317. TestTaskTrackerLocalization. (Ravi Gummadi via amareshwari)
  318. MAPREDUCE-2046. Fixes CombineFileInputFormat to allow splits with size
  319. less than DFS block size. (dhruba borthakur via amareshwari)
  320. MAPREDUCE-1975. Fixes unnecessary InterruptedException log in gridmix.
  321. (Ravi Gummadi via amareshwari)
  322. MAPREDUCE-1597. Fixes CombineFileInputFormat to work with non-splittable
  323. files. (amareshwari)
  324. MAPREDUCE-2032. Fixes TestJobCleanup to cleanup test directory in
  325. tearDown. (Dick King via amareshwari)
  326. MAPREDUCE-1979. Fixes "Output directory already exists" error in gridmix
  327. when gridmix.output.directory is not defined. (Ravi Gummadi via
  328. amareshwari)
  329. MAPREDUCE-1918. Adds documentation to Rumen. (Amar Kamat via amareshwari)
  330. MAPREDUCE-2078. Fixes TraceBuilder to generate traces when a globbed job
  331. history path is given. (Amar Kamat via amareshwari)
  332. MAPREDUCE-1989. Fixes error message in gridmix when user resolver is set
  333. and no user list is given. (Ravi Gummadi via amareshwari)
  334. MAPREDUCE-2067. Distinct minicluster services (e.g. NN and JT) overwrite
  335. each other's service policies. (Aaron T. Myers via tomwhite)
  336. MAPREDUCE-2029. DistributedRaidFileSystem removes itself from FileSystem
  337. cache when it is closed. (Ramkumar Vadali via dhruba)
  338. MAPREDUCE-1816. HAR files used for RAID parity-bite have configurable
  339. partfile size. (Ramkumar Vadali via dhruba)
  340. MAPREDUCE-2082. Fixes Pipes to create the jobtoken file in the right
  341. place. (Jitendra Pandey via ddas)
  342. MAPREDUCE-2095. Fixes Gridmix to run from compressed traces. (Ranjit
  343. Mathew via amareshwari)
  344. MAPREDUCE-1908. DistributedRaidFileSystem now handles ChecksumException
  345. correctly. (Ramkumar Vadali via schen)
  346. MAPREDUCE-2126. JobQueueJobInProgressListener's javadoc is inconsistent
  347. with source code. (Jingguo Yao via tomwhite)
  348. MAPREDUCE-2143. HarFileSystem is able to handle spaces in pathnames.
  349. (Ramkumar Vadali via dhruba)
  350. MAPREDUCE-1867. Remove unused methods in
  351. org.apache.hadoop.streaming.StreamUtil. (amareshwari via tomwhite)
  352. MAPREDUCE-2146. Raid does not affect access time of a source file.
  353. (Ramkumar Vadali via dhruba)
  354. MAPREDUCE-2150. RaidNode periodically fixes corrupt blocks. (Ramkumar Vadali via
  355. schen)
  356. MAPREDUCE-2099. RaidNode recreates outdated parity HARs. (Ramkumar Vadali
  357. via schen)
  358. MAPREDUCE-2173. Fix race condition in TestBlockFixer that was
  359. causing intermittent failure (Patrick Kling via dhruba)
  360. MAPREDUCE-2142. Refactor RaidNode so that the map-reduce component is
  361. clearly separated out. (Patrick Kling via dhruba)
  362. MAPREDUCE-2179. Fix RaidBlockSender compilation failure. (Ramkumar Vadali
  363. via schen)
  364. MAPREDUCE-2034. TestSubmitJob triggers NPE instead of permissions error.
  365. (Todd Lipcon via tomwhite)
  366. MAPREDUCE-2195. New property for local conf directory in
  367. system-test-mapreduce.xml file. (cos)
  368. MAPREDUCE-1783. FairScheduler initializes tasks only when the job can be
  369. run. (Ramkumar Vadali via schen)
  370. MAPREDUCE-2224. Fix synchronization bugs in JvmManager. (todd)
  371. MAPREDUCE-714. JobConf.findContainingJar unescapes unnecessarily on linux (todd)
  372. MAPREDUCE-2096. Secure local filesystem IO from symlink vulnerabilities (todd)
  373. MAPREDUCE-2234. If Localizer can't create task log directory, it should fail
  374. on the spot. (todd)
  375. MAPREDUCE-2219. JobTracker should not try to remove mapred.system.dir
  376. during startup. (todd)
  377. MAPREDUCE-2084. Remove deprecate annotation for package file. The package
  378. classes themselves are already deprecated. This removes an Eclipse error.
  379. (tomwhite via nigel)
  380. MAPREDUCE-1085. For tasks, "ulimit -v -1" is being run when user doesn't
  381. specify a ulimit (todd)
  382. MAPREDUCE-2282. Fix TestMRServerPorts for the changes in
  383. TestHDFSServerPorts. (shv via szetszwo)
  384. MAPREDUCE-2238. Fix permissions handling to avoid leaving undeletable
  385. directories in local dirs. (todd)
  386. MAPREDUCE-2277. TestCapacitySchedulerWithJobTracker needs to wait for jobs
  387. to complete before testing status. (todd)
  388. MAPREDUCE-2253. Servlets should specify content type (todd)
  389. MAPREDUCE-2283. Add timeout for Raid Tests (Ramkumar Vadali via schen)
  390. MAPREDUCE-1754. Replace mapred.persmissions.supergroup with an
  391. acl : mapreduce.cluster.administrators (Amareshwari Sriramadasu via shv)
  392. MAPREDUCE-2256. FairScheduler fairshare preemption from multiple pools may
  393. preempt all tasks from one pool causing that pool to go below fairshare.
  394. (Priyo Mustafi via shv)
  395. MAPREDUCE-2281. MR part of HADOOP-6642. (Chris Douglas, Po Cheung via shv)
  396. MAPREDUCE-2200. TestUmbilicalProtocolWithJobToken is failing without Krb
  397. environment: needs to be conditional. (Konstantin Boudnik via todd)
  398. MAPREDUCE-2077. Resolve name clash in the deprecated
  399. o.a.h.util.MemoryCalculatorPlugin (Luke Lu via shv)
  400. MAPREDUCE-2188. The new API MultithreadedMapper doesn't initialize
  401. RecordReader. (Owen O'Malley via shv)
  402. MAPREDUCE-1915. Fix IndexOutOfBoundsException in IndexCache.
  403. (Priyo Mustafi via shv)
  404. MAPREDUCE-1974. Fix multiple preemtions of the same task in FairScheduler.
  405. (Scott Chen via shv)
  406. MAPREDUCE-2304. Fix TestMRCLI to allow hostname with a hyphen (-).
  407. (Priyo Mustafi via shv)
  408. MAPREDUCE-1825. jobqueue_details.jsp and FairSchedulerServelet should not
  409. call finishedMaps and finishedReduces when job is not initialized.
  410. (Scott Chen via shv)
  411. MAPREDUCE-2285. MiniMRCluster does not start after ant test-patch
  412. (todd)
  413. MAPREDUCE-2311. Fix TestFairScheduler failure (schen)
  414. MAPREDUCE-2315. javadoc is failing in nightly build (todd)
  415. MAPREDUCE-2054. Hierarchical queue implementation broke dynamic queue
  416. addition in Dynamic Scheduler. (Thomas Sandholm via tomwhite)
  417. MAPREDUCE-2272. Job ACL file should not be executable
  418. (Harsh J Chouraria via todd)
  419. MAPREDUCE-2241. ClusterWithLinuxTaskController should accept relative path
  420. on the command line. (todd)
  421. MAPREDUCE-2251. Remove unused mapreduce.job.userhistorylocation config.
  422. (Harsh J Chouraria via todd)
  423. MAPREDUCE-2284. TestLocalRunner.testMultiMaps times out (todd)
  424. MAPREDUCE-2336. Tool-related packages should be in the Tool javadoc group.
  425. (tomwhite)
  426. MAPREDUCE-2394. JUnit output format doesn't propagate into raid contrib
  427. build. (todd)
  428. MAPREDUCE-2392. TaskTracker shutdown in the tests sometimes take 60s.
  429. (tomwhite)
  430. MAPREDUCE-2437. SLive uses only part* files to generating the final report.
  431. (shv)
  432. MAPREDUCE-2428. start-mapred.sh script fails if HADOOP_HOME is not set.
  433. (tomwhite via eli)
  434. MAPREDUCE-2445. Fix TestMiniMRWithDFSWithDistinctUsers to be a valid test.
  435. (todd)
  436. MAPREDUCE-2457. Job submission should inject group.name on the JobTracker
  437. (Alejandro Abdelnur via todd)
  438. MAPREDUCE-2448. NoSuchMethodError:
  439. org.apache.hadoop.hdfs.TestDatanodeBlockScanner.corruptReplica(..) (eli)
  440. MAPREDUCE-2472. Extra whitespace in mapred.child.java.opts breaks JVM
  441. initialization. (Aaron T. Myers via todd)
  442. MAPREDUCE-2222. Ivy resolve force mode should be turned off by default.
  443. (Luke Lu via tomwhite)
  444. MAPREDUCE-2486. Incorrect snapshot dependency published in .pom files
  445. (todd)
  446. MAPREDUCE-2327. MapTask doesn't need to put username information in
  447. SpillRecord. (todd via tomwhite)
  448. MAPREDUCE-2515. MapReduce code references some deprecated options
  449. (Ari Rabkin via todd)
  450. MAPREDUCE-2487. ChainReducer uses MAPPER_BY_VALUE instead of
  451. REDUCER_BY_VALUE. (Devaraj K via todd)
  452. MAPREDUCE-2571. CombineFileInputFormat.getSplits throws a
  453. java.lang.ArrayStoreException. (Bochun Bai via todd)
  454. MAPREDUCE-2127. mapreduce trunk builds are filing on hudson.
  455. (Bruno Mahé via eli)
  456. MAPREDUCE-2752. Build does not pass along properties to contrib builds.
  457. (Joep Rottinghuis via cos)
  458. MAPREDUCE-2753. Generated POMs hardcode dependency on hadoop-common
  459. version 0.22.0-SNAPSHOT (Joep Rottinghuis via cos)
  460. MAPREDUCE-2767. Remove Linux task-controller. (Milind Bhandarkar via shv)
  461. MAPREDUCE-2948. Hadoop streaming test failure, post MR-2767 (mahadev)
  462. MAPREDUCE-2940. Build fails with ant 1.7.0 but works with 1.8.0
  463. (Joep Rottinghuis via shv)
  464. MAPREDUCE-2991. queueinfo.jsp fails to show queue status for Capacity
  465. scheduler if queue names contain special symbols. (Priyo Mustafi via shv)
  466. MAPREDUCE-3025. Contribs not building. (Joep Rottinghuis via shv)
  467. MAPREDUCE-3026. Fix NPE in mapred queue -list with hierarchical queues.
  468. (Mayank Bansal via shv)
  469. MAPREDUCE-2779. JobSplitWriter.java can't handle large job.split file.
  470. (Ming Ma via shv)
  471. MAPREDUCE-2531. Fixed jobcontrol to downgrade JobID. (Robert Evans via
  472. acmurthy)
  473. MAPREDUCE-3151. Contrib tests failing (Joep Rottinghuis via cos)
  474. MAPREDUCE-3156. Allow TestMRCLI to be run against a cluster (cos)
  475. MAPREDUCE-2328. Add memory-related configuration parameters to
  476. mapred-default.xml. (Harsh J via shv)
  477. MAPREDUCE-3139. SlivePartitioner generates negative partitions. (jghoman)
  478. MAPREDUCE-1118. Enhance the JobTracker web-ui to ensure tabular columns
  479. are sortable, also added a /scheduler servlet to CapacityScheduler for
  480. enhanced UI for queue information.
  481. (Krishna Ramachandran and Milind Bhandarkar via shv)
  482. MAPREDUCE-3311. Bump jetty to 6.1.26 (cos)
  483. MAPREDUCE-3429. Few contrib tests are failing because of the missing
  484. commons-lang dependency (cos)
  485. MAPREDUCE-2059. RecoveryManager excludes jobtracker.info from the list of
  486. jobs to be recovered. (Subroto Sanyal via shv)
  487. MAPREDUCE-3438. TestRaidNode fails because of "Too many open files".
  488. (Ramkumar Vadali via shv)
  489. Release 0.21.1 - Unreleased
  490. NEW FEATURES
  491. MAPREDUCE-2040. Forrest Documentation for Dynamic Priority Scheduler.
  492. (Thomas Sandholm via tomwhite)
  493. BUG FIXES
  494. MAPREDUCE-1897. trunk build broken on compile-mapred-test (cos)
  495. MAPREDUCE-1280. Update Eclipse plugin to the new eclipse.jdt API.
  496. (Alex Kozlov via szetszwo)
  497. MAPREDUCE-1984. herriot TestCluster fails because exclusion is not there
  498. (Balaji Rajagopalan via cos)
  499. MAPREDUCE-2090. Clover build doesn't generate per-test coverage. (cos)
  500. MAPREDUCE-2134. ant binary-system is broken in mapreduce project. (cos)
  501. MAPREDUCE-1905. Fixes Context.setStatus() and progress() apis.
  502. (amareshwari)
  503. MAPREDUCE-1809. Ant build changes for Streaming system tests in contrib
  504. projects. (Vinay Kumar Thota via amareshwari)
  505. MAPREDUCE-2223. TestMRCLI might fail on Ubuntu with default /etc/hosts
  506. (cos)
  507. MAPREDUCE-2228. Remove java5 dependencies from build. (cos)
  508. MAPREDUCE-1929. Allow artifacts to be published to the staging Apache Nexus
  509. Maven Repository. (tomwhite)
  510. MAPREDUCE-2317. Fix a NPE in HadoopArchives. (Devaraj K via szetszwo)
  511. Release 0.21.0 - 2010-08-13
  512. INCOMPATIBLE CHANGES
  513. MAPREDUCE-516. Fix the starvation problem in the Capacity Scheduler
  514. when running High RAM Jobs. (Arun Murthy via yhemanth)
  515. MAPREDUCE-358. Change org.apache.hadoop.examples. AggregateWordCount
  516. and org.apache.hadoop.examples.AggregateWordHistogram to use new
  517. mapreduce api. (Amareshwari Sriramadasu via sharad)
  518. MAPREDUCE-245. Change Job and jobcontrol classes to use the List interface
  519. rather than ArrayList in APIs. (Tom White via cdouglas)
  520. MAPREDUCE-766. Enhanced list-blacklisted-trackers to display reasons
  521. for blacklisting a node. (Sreekanth Ramakrishnan via yhemanth)
  522. MAPREDUCE-817. Add a cache for retired jobs with minimal job info and
  523. provide a way to access history file url. (sharad)
  524. MAPREDUCE-711. Moved Distributed Cache from Common to Map/Reduce
  525. project. (Vinod Kumar Vavilapalli via yhemanth)
  526. MAPREDUCE-895. Per the contract elucidated in HADOOP-6201, throw
  527. FileNotFoundException from FileSystem::listStatus rather than returning
  528. null. (Jakob Homan via cdouglas)
  529. MAPREDUCE-479. Provide full task id to map output servlet rather than the
  530. reduce id, only. (Jiaqi Tan via cdouglas)
  531. MAPREDUCE-873. Simplify job recovery. Incomplete jobs are resubmitted on
  532. jobtracker restart. Removes a public constructor in JobInProgress. (sharad)
  533. HADOOP-6230. Moved process tree and memory calculator related classes from
  534. Common to Map/Reduce. (Vinod Kumar Vavilapalli via yhemanth)
  535. MAPREDUCE-157. Refactor job history APIs and change the history format to
  536. JSON. (Jothi Padmanabhan via sharad)
  537. MAPREDUCE-849. Rename configuration properties. (Amareshwari Sriramadasu
  538. via sharad)
  539. MAPREDUCE-1287. Only call the partitioner with more than one reducer.
  540. (cdouglas)
  541. MAPREDUCE-1385. Use the new UserGroupInformation from HADOOP-6299.
  542. (ddas via omalley)
  543. MAPREDUCE-1493. Authorization for job-history pages. (vinodkv)
  544. MAPREDUCE-1607. Task controller may not set permissions for a
  545. task cleanup attempt's log directory (Amareshwari Sriramadasu via vinodkv)
  546. MAPREDUCE-1683. Remove JNI calls from ClusterStatus cstr. (Arun Murthy and
  547. Luke Lu via cdouglas)
  548. MAPREDUCE-1855. Makes the refresh methods (for groups and proxy users)
  549. independent of the client side configuration. (Boris Shkolnik via ddas)
  550. NEW FEATURES
  551. MAPREDUCE-1774. Large-scale Automated Framework (Sharad Agarwal, Sreekanth
  552. Ramakrishnan, Konstantin Boudnik, at all via cos)
  553. MAPREDUCE-706. Support for FIFO pools in the fair scheduler.
  554. (Matei Zaharia)
  555. MAPREDUCE-546. Provide sample fair scheduler config file in conf/ and use
  556. it by default if no other config file is specified. (Matei Zaharia)
  557. MAPREDUCE-551. Preemption support in the Fair Scheduler. (Matei Zaharia)
  558. MAPREDUCE-567. Add a new example MR that always fails. (Philip Zeyliger
  559. via tomwhite)
  560. MAPREDUCE-211. Provides ability to run a health check script on the
  561. tasktracker nodes and blacklist nodes if they are unhealthy.
  562. (Sreekanth Ramakrishnan via yhemanth)
  563. MAPREDUCE-637. Add an example, distbbp, which able to compute the n th bit
  564. of Pi for some large n. (szetszwo)
  565. MAPREDUCE-532. Provide a way to limit the number of used slots
  566. per queue in the capacity scheduler.
  567. (Rahul Kumar Singh via yhemanth)
  568. MAPREDUCE-467. Provide ability to collect statistics about total tasks
  569. and succeeded tasks in different time windows. (sharad)
  570. MAPREDUCE-740. Log a job-summary at the end of a job, while allowing it
  571. to be configured to use a custom appender if desired. (acmurthy)
  572. MAPREDUCE-814. Provide a way to configure completed job history files
  573. to be on HDFS. (sharad)
  574. MAPREDUCE-800. MRUnit should support the new API. (Aaron Kimball via
  575. tomwhite)
  576. MAPREDUCE-798. MRUnit should be able to test a succession of MapReduce
  577. passes. (Aaron Kimball via tomwhite)
  578. MAPREDUCE-768. Provide an option to dump jobtracker configuration in JSON
  579. format to standard output. (V.V.Chaitanya Krishna via yhemanth)
  580. MAPREDUCE-824. Add support for a hierarchy of queues in the capacity
  581. scheduler. (Rahul Kumar Singh via yhemanth)
  582. MAPREDUCE-751. Add Rumen, a tool for extracting statistics from job tracker
  583. logs and generating job traces for simulation and analysis. (Dick King via
  584. cdouglas)
  585. MAPREDUCE-830. Add support for splittable compression to TextInputFormats.
  586. (Abdul Qadeer via cdouglas)
  587. MAPREDUCE-861. Add support for hierarchical queues in the Map/Reduce
  588. framework. (Rahul Kumar Singh via yhemanth)
  589. MAPREDUCE-776. Add Gridmix, a benchmark processing Rumen traces to simulate
  590. a measured mix of jobs on a cluster. (cdouglas)
  591. MAPREDUCE-862. Enhance JobTracker UI to display hierarchical queues.
  592. (V.V.Chaitanya Krishna via yhemanth)
  593. MAPREDUCE-777. Brand new apis to track and query jobs as a
  594. replacement for JobClient. (Amareshwari Sriramadasu via acmurthy)
  595. MAPREDUCE-775. Add native and streaming support for Vertica as an input
  596. or output format taking advantage of parallel read and write properties of
  597. the DBMS. (Omer Trajman via ddas)
  598. MAPREDUCE-679. XML-based metrics as JSP servlet for JobTracker.
  599. (Aaron Kimball via tomwhite)
  600. MAPREDUCE-980. Modify JobHistory to use Avro for serialization. (cutting)
  601. MAPREDUCE-728. Add Mumak, a Hadoop map/reduce simulator. (Arun C Murthy,
  602. Tamas Sarlos, Anirban Dasgupta, Guanying Wang, and Hong Tang via cdouglas)
  603. MAPREDUCE-1383. Automates fetching of delegation tokens in File*Formats
  604. Distributed Cache and Distcp. Also, provides a config
  605. mapreduce.job.hdfs-servers that the jobs can populate with a comma
  606. separated list of namenodes. The job client automatically fetches
  607. delegation tokens from those namenodes. (Boris Shkolnik via ddas)
  608. MAPREDUCE-698. Per-pool task limits for the fair scheduler.
  609. (Kevin Peterson via matei)
  610. MAPREDUCE-1026. Does mutual authentication of the shuffle
  611. transfers using a shared JobTracker generated key.
  612. (Boris Shkolnik via ddas)
  613. MAPREDUCE-744. Introduces the notion of a public distributed cache.
  614. (Devaraj Das)
  615. MAPREDUCE-1338. Introduces the notion of token cache using which
  616. tokens and secrets can be sent by the Job client to the JobTracker.
  617. (Boris Shkolnik via ddas)
  618. HDFS-503. This patch implements an optional layer over HDFS that
  619. implements offline erasure-coding. It can be used to reduce the
  620. total storage requirements of HDFS. (dhruba)
  621. MAPREDUCE-1432. Adds hooks in the jobtracker and tasktracker
  622. for loading the tokens in the user's ugi. This is required
  623. for the copying of files from the hdfs. (ddas)
  624. MAPREDUCE-1335. Adds SASL Kerberos/Digest authentication in MapReduce.
  625. (Kan Zhang via ddas)
  626. MAPREDUCE-1464. Makes a compatible change in JobTokenIdentifier to
  627. account for HADOOP-6510. (Jitendra Nath Pandey via ddas)
  628. MAPREDUCE-1433. Add a delegation token for MapReduce. (omalley)
  629. MAPREDUCE-1307. Introduces the Job level ACLs feature.
  630. (Vinod Kumar Vavilapalli via ddas)
  631. MAPREDUCE-1430. JobTracker automatically renews delegation tokens for jobs.
  632. (Boris Shkolnik via ddas)
  633. MAPREDUCE-1455. Introduces job-level authorization for mapreduce servlets.
  634. (Ravi Gummadi via vinodkv)
  635. IMPROVEMENTS
  636. MAPREDUCE-463. Makes job setup and cleanup tasks as optional.
  637. (Amareshwari Sriramadasu via sharad)
  638. MAPREDUCE-502. Allow jobtracker to be configured with zero completed jobs
  639. in memory. (Amar Kamat via sharad)
  640. MAPREDUCE-416. Moves the history file to a "done" folder whenever a job
  641. completes. (Amar Kamat via ddas)
  642. MAPREDUCE-646. Increase srcfilelist replication number in dictcp job.
  643. (Ravi Gummadi via szetszwo)
  644. HADOOP-6106. Updated hadoop-core and test jars from hudson trunk
  645. build #12. (Giridharan Kesavan)
  646. MAPREDUCE-642. A option to distcp that allows preserving the full
  647. source path of a file in the specified destination directory.
  648. (Rodrigo Schmidt via dhruba)
  649. MAPREDUCE-686. Move TestSpeculativeExecution.Fake* into a separate class
  650. so that it can be used by other tests. (Jothi Padmanabhan via sharad)
  651. MAPREDUCE-625. Modify TestTaskLimits to improve execution time.
  652. (Jothi Padmanabhan via sharad)
  653. MAPREDUCE-465. Deprecate o.a.h.mapred.lib.MultithreadedMapRunner and add
  654. test for o.a.h.mapreduce.lib.MultithreadedMapper.
  655. (Amareshwari Sriramadasu via sharad)
  656. MAPREDUCE-701. Improves the runtime of the TestRackAwareTaskPlacement
  657. by making it a unit test. (Jothi Padmanabhan via ddas)
  658. MAPREDUCE-371. Change KeyFieldBasedComparator and KeyFieldBasedPartitioner
  659. to use new api. (Amareshwari Sriramadasu via sharad)
  660. MAPREDUCE-623. Resolve javac warnings in mapreduce. (Jothi Padmanabhan
  661. via sharad)
  662. MAPREDUCE-655. Change KeyValueLineRecordReader and KeyValueTextInputFormat
  663. to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
  664. MAPREDUCE-632. Merge TestCustomOutputCommitter with
  665. TestCommandLineJobSubmission. (Jothi Padmanabhan via sharad)
  666. MAPREDUCE-627. Improves execution time of TestTrackerBlacklistAcrossJobs.
  667. (Jothi Padmanabhan via ddas)
  668. MAPREDUCE-630. Improves execution time of TestKillCompletedJob.
  669. (Jothi Padmanabhan via ddas)
  670. MAPREDUCE-626. Improves the execution time of TestLostTracker.
  671. (Jothi Padmanabhan via ddas)
  672. MAPREDUCE-353. Makes the shuffle read and connection timeouts
  673. configurable. (Ravi Gummadi via ddas)
  674. MAPREDUCE-739. Allow relative paths to be created in archives. (Mahadev
  675. Konar via cdouglas)
  676. MAPREDUCE-772. Merge HADOOP-4010 changes to LineRecordReader into mapreduce
  677. package. (Abdul Qadeer via cdouglas)
  678. MAPREDUCE-785. Separate sub-test of TestReduceFetch to be included in
  679. MR-670. (Jothi Padmanabhan via cdouglas)
  680. MAPREDUCE-784. Modify TestUserDefinedCounters to use LocalJobRunner
  681. instead of MiniMR. (Jothi Padmanabhan via sharad)
  682. HADOOP-6160. Fix releaseaudit target to run on specific directories.
  683. (gkesavan)
  684. MAPREDUCE-782. Use PureJavaCrc32 in SpillRecord. (Todd Lipcon via
  685. szetszwo)
  686. MAPREDUCE-369. Change org.apache.hadoop.mapred.lib.MultipleInputs to
  687. use new api. (Amareshwari Sriramadasu via sharad)
  688. MAPREDUCE-373. Change org.apache.hadoop.mapred.lib.FieldSelectionMapReduce
  689. to use new api. (Amareshwari Sriramadasu via sharad)
  690. MAPREDUCE-628. Improves the execution time of TestJobInProgress.
  691. (Jothi Padmanabhan via ddas)
  692. MAPREDUCE-793. Creates a new test that consolidates a few tests to
  693. include in the commit-test list. (Jothi Padmanabhan via ddas)
  694. MAPREDUCE-797. Adds combiner support to MRUnit MapReduceDriver.
  695. (Aaron Kimball via johan)
  696. MAPREDUCE-656. Change org.apache.hadoop.mapred.SequenceFile* classes
  697. to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
  698. MAPREDUCE-670. Creates ant target for 10 mins patch test build.
  699. (Jothi Padmanabhan via gkesavan)
  700. MAPREDUCE-375. Change org.apache.hadoop.mapred.lib.NLineInputFormat
  701. and org.apache.hadoop.mapred.MapFileOutputFormat to use new api.
  702. (Amareshwari Sriramadasu via ddas)
  703. MAPREDUCE-779. Added node health failure counts into
  704. JobTrackerStatistics. (Sreekanth Ramakrishnan via yhemanth)
  705. MAPREDUCE-842. Setup secure permissions for localized job files,
  706. intermediate outputs and log files on tasktrackers.
  707. (Vinod Kumar Vavilapalli via yhemanth)
  708. MAPREDUCE-478. Allow map and reduce jvm parameters, environment variables
  709. and ulimit to be set separately.
  710. Configuration changes:
  711. add mapred.map.child.java.opts
  712. add mapred.reduce.child.java.opts
  713. add mapred.map.child.env
  714. add mapred.reduce.child.ulimit
  715. add mapred.map.child.env
  716. add mapred.reduce.child.ulimit
  717. deprecated mapred.child.java.opts
  718. deprecated mapred.child.env
  719. deprecated mapred.child.ulimit
  720. (acmurthy)
  721. MAPREDUCE-767. Remove the dependence on the CLI 2.0 snapshot.
  722. (Amar Kamat via omalley)
  723. MAPREDUCE-712. Minor efficiency tweaks to RandomTextWriter. (cdouglas)
  724. MAPREDUCE-870. Remove the job retire thread and the associated
  725. config parameters. (sharad)
  726. MAPREDUCE-874. Rename the PiEstimator example to QuasiMonteCarlo.
  727. (szetszwo)
  728. MAPREDUCE-336. Allow logging level of map/reduce tasks to be configurable.
  729. Configuration changes:
  730. add mapred.map.child.log.level
  731. add mapred.reduce.child.log.level
  732. (acmurthy)
  733. MAPREDUCE-355. Update mapred.join package to use the new API. (Amareshwari
  734. Sriramadasu via cdouglas)
  735. HADOOP-6184. Updated hadoop common and test jars to get the new API
  736. in Configuration for dumping in JSON format from Hudson trunk build #68.
  737. (yhemanth)
  738. MAPREDUCE-476. Extend DistributedCache to work locally (LocalJobRunner).
  739. (Philip Zeyliger via tomwhite)
  740. MAPREDUCE-825. JobClient completion poll interval of 5s causes slow tests
  741. in local mode. (Aaron Kimball via tomwhite)
  742. MAPREDUCE-910. Support counters in MRUnit. (Aaron Kimball via cdouglas)
  743. MAPREDUCE-788. Update gridmix2 to use the new API (Amareshwari Sriramadasu
  744. via cdouglas)
  745. MAPREDUCE-875. Make DBRecordReader execute queries lazily. (Aaron Kimball
  746. via enis)
  747. MAPREDUCE-318. Modularizes the shuffle code. (Jothi Padmanabhan and
  748. Arun Murthy via ddas)
  749. MAPREDUCE-936. Allow a load difference for fairshare scheduler.
  750. (Zheng Shao via dhruba)
  751. MAPREDUCE-370. Update MultipleOutputs to use the API, merge funcitonality
  752. of MultipleOutputFormat. (Amareshwari Sriramadasu via cdouglas)
  753. MAPREDUCE-898. Changes DistributedCache to use the new API.
  754. (Amareshwari Sriramadasu via ddas)
  755. MAPREDUCE-144. Includes dump of the process tree in task diagnostics when
  756. a task is killed due to exceeding memory limits.
  757. (Vinod Kumar Vavilapalli via yhemanth)
  758. MAPREDUCE-945. Modifies MRBench and TestMapRed to use ToolRunner so that
  759. options such as queue name can be passed via command line.
  760. (Sreekanth Ramakrishnan via yhemanth)
  761. MAPREDUCE-963. Deprecate o.a.h.mapred.FileAlreadyExistsException and
  762. replace it with o.a.h.fs.FileAlreadyExistsException. (Boris Shkolnik
  763. via szetszwo)
  764. MAPREDUCE-960. Remove an unnecessary intermediate copy and obsolete API
  765. from KeyValueLineRecordReader. (cdouglas)
  766. MAPREDUCE-930. Modify Rumen to resolve paths in the canonical way, rather
  767. than defaulting to the local filesystem. (cdouglas)
  768. MAPREDUCE-944. Extend the LoadManager API of the fair-share scheduler
  769. to support regulating tasks for a job based on resources currently in use
  770. by that job. (dhruba)
  771. MAPREDUCE-973. Move FailJob and SleepJob from examples to test. (cdouglas
  772. via omalley)
  773. MAPREDUCE-966. Modify Rumen to clean up interfaces and simplify integration
  774. with other tools. (Hong Tang via cdouglas)
  775. MAPREDUCE-856. Setup secure permissions for distributed cache files.
  776. (Vinod Kumar Vavilapalli via yhemanth)
  777. MAPREDUCE-885. More efficient SQL queries for DBInputFormat. (Aaron Kimball
  778. via enis)
  779. MAPREDUCE-284. Enables ipc.client.tcpnodelay in Tasktracker's Child.
  780. (Ravi Gummadi via sharad)
  781. MAPREDUCE-916. Split the documentation to match the project split.
  782. (Corinne Chandel via omalley)
  783. MAPREDUCE-649. Validate a copy by comparing the source and destination
  784. checksums in distcp. Also adds an intra-task retry mechanism for errors
  785. detected during the copy. (Ravi Gummadi via cdouglas)
  786. MAPREDUCE-654. Add a -dryrun option to distcp printing a summary of the
  787. file data to be copied, without actually performing the copy. (Ravi Gummadi
  788. via cdouglas)
  789. MAPREDUCE-664. Display the number of files deleted by distcp when the
  790. -delete option is specified. (Ravi Gummadi via cdouglas)
  791. MAPREDUCE-781. Let the name of distcp jobs be configurable. (Venkatesh S
  792. via cdouglas)
  793. MAPREDUCE-975. Add an API in job client to get the history file url for
  794. a given job id. (sharad)
  795. MAPREDUCE-905. Add Eclipse launch tasks for MapReduce. (Philip Zeyliger
  796. via tomwhite)
  797. MAPREDUCE-277. Makes job history counters available on the job history
  798. viewers. (Jothi Padmanabhan via ddas)
  799. MAPREDUCE-893. Provides an ability to refresh queue configuration
  800. without restarting the JobTracker.
  801. (Vinod Kumar Vavilapalli and Rahul Kumar Singh via yhemanth)
  802. MAPREDUCE-1011. Add build.properties to svn and git ignore. (omalley)
  803. MAPREDUCE-954. Change Map-Reduce context objects to be interfaces.
  804. (acmurthy)
  805. MAPREDUCE-639. Change Terasort example to reflect the 2009 updates.
  806. (omalley)
  807. MAPREDUCE-1063. Document gridmix benchmark. (cdouglas)
  808. MAPREDUCE-931. Use built-in interpolation classes for making up task
  809. runtimes in Rumen. (Dick King via cdouglas)
  810. MAPREDUCE-1012. Mark Context interfaces as public evolving. (Tom White via
  811. cdouglas)
  812. MAPREDUCE-971. Document use of distcp when copying to s3, managing timeouts
  813. in particular. (Aaron Kimball via cdouglas)
  814. HDFS-663. DFSIO for append. (shv)
  815. HDFS-641. Move all of the components that depend on map/reduce to
  816. map/reduce. (omalley)
  817. HADOOP-5107. Use Maven ant tasks to publish artifacts. (Giridharan Kesavan
  818. via omalley)
  819. MAPREDUCE-1229. Allow customization of job submission policy in Mumak.
  820. (Hong Tang via cdouglas)
  821. MAPREDUCE-1317. Reduce the memory footprint of Rumen objects by interning
  822. host Strings. (Hong Tang via cdouglas)
  823. MAPREDUCE-1097. Add support for Vertica 3.5 to its contrib module. (Omer
  824. Trajman via cdouglas)
  825. MAPREDUCE-1627. HadoopArchives should not uses a method in DistCp.
  826. (szetszwo)
  827. MAPREDUCE-1198. Alternatively schedule different types of tasks in
  828. fair share scheduler. (Scott Chen via matei)
  829. MAPREDUCE-707. Provide a jobconf property for explicitly assigning a job to
  830. a pool in the Fair Scheduler. (Alan Heirich via matei)
  831. MAPREDUCE-947. Added commitJob and abortJob apis to OutputCommitter.
  832. Enhanced FileOutputCommitter to create a _SUCCESS file for successful
  833. jobs. (Amar Kamat & Jothi Padmanabhan via acmurthy)
  834. MAPREDUCE-1103. Added more metrics to Jobtracker. (sharad)
  835. MAPREDUCE-1048. Add occupied/reserved slot usage summary on jobtracker UI.
  836. (Amareshwari Sriramadasu and Hemanth Yamijala via sharad)
  837. MAPREDUCE-1090. Modified log statement in TaskMemoryManagerThread to
  838. include task attempt id. (yhemanth)
  839. MAPREDUCE-1189. Reduce ivy console output to ovservable level (cos)
  840. MAPREDUCE-1167. ProcfsBasedProcessTree collects rss memory information.
  841. (Scott Chen via dhruba)
  842. MAPREDUCE-1231. Added a new DistCp option, -skipcrccheck, so that the CRC
  843. check during setup can be skipped. (Jothi Padmanabhan via szetszwo)
  844. MAPREDUCE-1190. Add package documentation for BBP example.
  845. (Tsz Wo (Nicholas) Sze via cdouglas)
  846. MAPREDUCE-1119. When tasks fail to report status, show tasks's stack dump
  847. before killing. (Aaron Kimball via tomwhite)
  848. MAPREDUCE-1185. Redirect running job url to history url if job is already
  849. retired. (Amareshwari Sriramadasu and Sharad Agarwal via sharad)
  850. MAPREDUCE-1050. Introduce a mock object testing framework. (tomwhite)
  851. MAPREDUCE-1084. Implementing aspects development and fault injeciton
  852. framework for MapReduce. (Sreekanth Ramakrishnan via cos)
  853. MAPREDUCE-1209. Move common specific part of the test TestReflectionUtils
  854. out of mapred into common. (Todd Lipcon via tomwhite)
  855. MAPREDUCE-967. TaskTracker does not need to fully unjar job jars.
  856. (Todd Lipcon via tomwhite)
  857. MAPREDUCE-1083. Changes in MapReduce so that group information of users
  858. can be refreshed in the JobTracker via command line.
  859. (Boris Shkolnik via ddas)
  860. MAPREDUCE-181. Changes the job submission process to be secure.
  861. (Devaraj Das)
  862. MAPREDUCE-1250. Refactors the JobToken to use Common's Token interface.
  863. (Kan Zhang via ddas)
  864. MAPREDUCE-896. Enhance tasktracker to cleanup files that might have
  865. been created by user tasks with non-writable permissions.
  866. (Ravi Gummadi via yhemanth)
  867. MAPREDUCE-372. Change org.apache.hadoop.mapred.lib.ChainMapper/Reducer
  868. to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
  869. MAPREDUCE-1295. Add a tool in Rumen for folding and manipulating job
  870. traces. (Dick King via cdouglas)
  871. MAPREDUCE-1302. TrackerDistributedCacheManager deletes file
  872. asynchronously, thus reducing task initialization delays.
  873. (Zheng Shao via dhruba)
  874. MAPREDUCE-1218. TaskTrackers send cpu and memory usage of
  875. node to JobTracker. (Scott Chen via dhruba)
  876. MAPREDUCE-847. Fix Releaseaudit warning count to zero
  877. (Giridharan Kesavan)
  878. MAPREDUCE-1337. Use generics in StreamJob to improve readability of that
  879. class. (Kay Kay via cdouglas)
  880. MAPREDUCE-361. Port terasort example to the new mapreduce API. (Amareshwari
  881. Sriramadasu via cdouglas)
  882. MAPREDUCE-1367. LocalJobRunner should support parallel mapper execution.
  883. (Aaron Kimball via tomwhite)
  884. MAPREDUCE-64. Eliminate io.sort.record.percent from MapTask configuration.
  885. (cdouglas)
  886. MAPREDUCE-1440. Replace the long user name in MapReduce with the local
  887. name. (omalley)
  888. MAPREDUCE-1470. Move delegation tokens from HDFS to Common so that
  889. MapReduce can use them too. (omalley)
  890. MAPREDUCE-1425. Reduce memory usage by archive. (mahadev via szetszwo)
  891. MAPREDUCE-1441. Trim whitespace from directory lists pulled from the
  892. configuration. (Todd Lipcon via cdouglas)
  893. MAPREDUCE-1309. Refactor Rumen trace generator to improve code structure
  894. and add extensible support for log formats. (Dick King via cdouglas)
  895. MAPREDUCE-1503. Delegation token renewing and cancelling should provide
  896. meaningful exceptions when there are failures instead of returning
  897. false. (omalley)
  898. HADOOP-6579. Upgrade commons-codec library to 1.4. (omalley)
  899. MAPREDUCE-1423. Improve performance of CombineFileInputFormat when multiple
  900. pools are configured. (Dhruba Borthakur via zshao)
  901. MAPREDUCE-1454. Quote user supplied strings in Tracker servlets. (cdouglas)
  902. MAPREDUCE-1408. Add customizable job submission policies to Gridmix. (Rahul
  903. Singh via cdouglas)
  904. MAPREDUCE-1527. Better warning logged when mapred.queue.names is
  905. overshadowed by mapred-queues.xml. (Hong Tang via acmurthy)
  906. MAPREDUCE-1403. Save the size and number of distributed cache artifacts in
  907. the configuration. (Arun Murthy via cdouglas)
  908. MAPREDUCE-1482. Truncate state string and diagnostic information in
  909. TaskStatus. (Amar Kamat via szetszwo)
  910. MAPREDUCE-1593. [Rumen] Improvements to random seed generation (tamas via
  911. mahadev)
  912. MAPREDUCE-1460. Oracle support in DataDrivenDBInputFormat.
  913. (Aaron Kimball via tomwhite)
  914. MAPREDUCE-1569. Pass configuration through mocked contexts in MRUnit.
  915. (Chris White via cdouglas)
  916. MAPREDUCE-1590. Move HarFileSystem from Hadoop Common to Mapreduce tools.
  917. (mahadev)
  918. MAPREDUCE-1629. Get rid of fakeBlockLocations() on HarFileSystem, since
  919. it's not used (mahadev)
  920. MAPREDUCE-1489. DataDrivenDBInputFormat should not query the database
  921. when generating only one split. (Aaron Kimball via tomwhite)
  922. MAPREDUCE-1514. Add documentation on replication, permissions, new options,
  923. limitations and internals of har. (mahadev via szetszwo)
  924. MAPREDUCE-1428. Make block size and the size of archive created files
  925. configurable. (mahadev via szetszwo)
  926. MAPREDUCE-1656. JobStory should provide queue info. (hong via mahadev)
  927. MAPREDUCE-1466. Record number of files processed in FileInputFormat in the
  928. Configuration for offline analysis. (Luke Lu and Arun Murthy via cdouglas)
  929. MAPREDUCE-1538. TrackerDistributedCacheManager manages the
  930. number of files. (Scott Chen via dhruba)
  931. MAPREDUCE-1673. Scripts to start and stop RaidNode.
  932. (Rodrigo Schmidt via dhruba)
  933. MAPREDUCE-1659. RaidNode writes temp files on configured tmp directory and
  934. add random numbers to their names to avoid conflicts
  935. (Rodrigo Schmidt via dhruba)
  936. MAPREDUCE-1221. Allow admins to control physical memory limits per-task
  937. and per-node. (Scott Chen via acmurthy)
  938. MAPREDUCE-1065. Update mapred tutorial to use the new API. (Aaron Kimball
  939. via cdouglas)
  940. MAPREDUCE-1304. Add a task counter tracking time spent in GC. (Aaron
  941. Kimball via cdouglas)
  942. MAPREDUCE-1570. Add grouping comparators to MRUnit. (Chris White via
  943. cdouglas)
  944. MAPREDUCE-1650. Exclude Private elements from generated MapReduce
  945. Javadoc. (tomwhite)
  946. MAPREDUCE-1625. Improve grouping of packages in Javadoc. (tomwhite)
  947. MAPREDUCE-1417. Forrest documentation should be updated to reflect
  948. the changes in MAPREDUCE-744. (Ravi Gummadi via vinodkv)
  949. MAPREDUCE-1568. TrackerDistributedCacheManager should clean up cache
  950. in a background thread. (Scott Chen via zshao)
  951. MAPREDUCE-1749. Move configuration strings out of JobContext so that it
  952. can be made public stable. (omalley)
  953. MAPREDUCE-1623. Apply audience and stability notations to Hadoop
  954. Map-Reduce. (tomwhite via acmurthy)
  955. MAPREDUCE-1751. Change MapReduce to depend on Hadoop 'common' artifacts
  956. instead of 'core'. (tomwhite)
  957. MAPREDUCE-1535. Replace usage of FileStatus#isDir(). (Eli Collins via
  958. tomwhite)
  959. MAPREDUCE-1832. Allow file sizes less than 1MB in DFSIO benchmark. (shv)
  960. MAPREDUCE-1404. Move Cluster-Setup and Single-Node-Setup Docs from
  961. MapReduce to Common. (tomwhite)
  962. MAPREDUCE-1697. Document the behavior of -file option and deprecate it
  963. in favour of -files option in streaming. (Amareshwari Sriramadasu
  964. via vinodkv)
  965. MAPREDUCE-1033. Resolve location of scripts and configuration files after
  966. project split. (tomwhite)
  967. MAPREDUCE-1018. Document changes to the memory management and scheduling
  968. model. (Hemanth Yamijala via vinodkv)
  969. MAPREDUCE-1896. [Herriot] New property for multi user list. (Vinay Thota
  970. via cos)
  971. MAPREDUCE-1812. New properties for suspend and resume process. (Vinay
  972. Thota via cos)
  973. OPTIMIZATIONS
  974. MAPREDUCE-270. Fix the tasktracker to optionally send an out-of-band
  975. heartbeat on task-completion for better job-latency. (acmurthy)
  976. Configuration changes:
  977. add mapreduce.tasktracker.outofband.heartbeat
  978. MAPREDUCE-1186. Modified code in distributed cache to set permissions
  979. only on required set of localized paths.
  980. (Amareshwari Sriramadasu via yhemanth)
  981. MAPREDUCE-1501. FileInputFormat supports multi-level, recursive
  982. directory listing. (Zheng Shao via dhruba)
  983. MAPREDUCE-1556. upgrade to Avro 1.3.0. (cutting via tomwhite)
  984. MAPREDUCE-1613. Install/deploy source jars to Maven repo
  985. (Patrick Angeles via ddas)
  986. MAPREDUCE-1610. Forrest documentation should be updated to reflect
  987. the changes in MAPREDUCE-856. (Ravi Gummadi via vinodkv)
  988. MAPREDUCE-1853. Adds caching for TaskAttemptContext in MultipleOutputs.
  989. (Torsten Curdt via amareshwari)
  990. BUG FIXES
  991. MAPREDUCE-878. Rename fair scheduler design doc to
  992. fair-scheduler-design-doc.tex and add Apache license header (matei)
  993. HADOOP-4687. MapReduce is split from Hadoop Core. It is a subproject under
  994. Hadoop (Owen O'Malley)
  995. HADOOP-6096. Fix Eclipse project and classpath files following project
  996. split. (tomwhite)
  997. MAPREDUCE-419. Reconcile mapred.userlog.limit.kb defaults in configuration
  998. and code. (Philip Zeyliger via cdouglas)
  999. MAPREDUCE-2. Fixes a bug in KeyFieldBasedPartitioner in handling empty
  1000. keys. (Amar Kamat via sharad)
  1001. MAPREDUCE-130. Delete the jobconf copy from the log directory of the
  1002. JobTracker when the job is retired. (Amar Kamat via sharad)
  1003. MAPREDUCE-657. Fix hardcoded filesystem problem in CompletedJobStatusStore.
  1004. (Amar Kamat via sharad)
  1005. MAPREDUCE-179. Update progress in new RecordReaders. (cdouglas)
  1006. MAPREDUCE-658. Replace NPE in distcp with a meaningful error message when
  1007. the source path does not exist. (Ravi Gummadi via cdouglas)
  1008. MAPREDUCE-671. Update ignore list to include untracked, generated
  1009. build artifacts and config files. (cdouglas)
  1010. MAPREDUCE-433. Use more reliable counters in TestReduceFetch. (cdouglas)
  1011. MAPREDUCE-124. Fix a bug in failure handling of abort task of
  1012. OutputCommiter. (Amareshwari Sriramadasu via sharad)
  1013. MAPREDUCE-694. Fix to add jsp-api jars to capacity-scheduler classpath.
  1014. (Giridharan Kesavan)
  1015. MAPREDUCE-702. Fix eclipse-plugin jar target (Giridharan Kesavan)
  1016. MAPREDUCE-522. Replace TestQueueCapacities with simpler test case to
  1017. test integration between capacity scheduler and MR framework.
  1018. (Sreekanth Ramakrishnan via yhemanth)
  1019. MAPREDUCE-683. Fixes an initialization problem in the JobHistory.
  1020. The initialization of JobHistoryFilesManager is now done in the
  1021. JobHistory.init call. (Amar Kamat via ddas)
  1022. MAPREDUCE-708. Fixes a bug to allow updating the reason for
  1023. blacklisting a node on the JobTracker UI.
  1024. (Sreekanth Ramakrishnan via yhemanth)
  1025. MAPREDUCE-709. Fixes message displayed for a blacklisted node where
  1026. the reason for blacklisting is due to the health check script
  1027. timing out. (Sreekanth Ramakrishnan via yhemanth)
  1028. MAPREDUCE-676. Existing diagnostic rules fail for MAP ONLY jobs.
  1029. (Suhas Gogate via tomwhite)
  1030. MAPREDUCE-722. Fixes a bug with tasktracker reservations for
  1031. high memory jobs in capacity scheduler.
  1032. (Vinod Kumar Vavilapalli via yhemanth)
  1033. HADOOP-6090. Updates gridmix script to use new mapreduce api output
  1034. format. (Amareshwari Sriramadasu via sharad)
  1035. MAPREDUCE-732. Removed spurious log statements in the node
  1036. blacklisting logic. (Sreekanth Ramakrishnan via yhemanth)
  1037. MAPREDUCE-734. Fix a ConcurrentModificationException in unreserving
  1038. unused reservations for a job when it completes.
  1039. (Arun Murthy and Sreekanth Ramakrishnan via yhemanth)
  1040. MAPREDUCE-733. Fix a RuntimeException while unreserving trackers
  1041. that are blacklisted for a job.
  1042. (Arun Murthy and Sreekanth Ramakrishnan via yhemanth)
  1043. MAPREDUCE-677. Fix timeout in TestNodeRefresh. (Amar Kamat via
  1044. sharad)
  1045. MAPREDUCE-153. Fix timeout in TestJobInProgressListener. (Amar
  1046. Kamat via sharad)
  1047. MAPREDUCE-742. Fix output messages and java comments in the Pi related
  1048. examples. (szetszwo)
  1049. MAPREDUCE-565. Fix partitioner to work with new API. (Owen O'Malley via
  1050. cdouglas)
  1051. MAPREDUCE-680. Fix so MRUnit can handle reuse of Writable objects.
  1052. (Aaron Kimball via johan)
  1053. MAPREDUCE-18. Puts some checks for cross checking whether a reduce
  1054. task gets the correct shuffle data. (Ravi Gummadi via ddas)
  1055. MAPREDUCE-771. Fix scheduling of setup and cleanup tasks to use
  1056. free slots instead of tasks for scheduling. (yhemanth)
  1057. MAPREDUCE-717. Fixes some corner case issues in speculative
  1058. execution heuristics. (Devaraj Das)
  1059. MAPREDUCE-716. Make DBInputFormat work with Oracle. (Aaron Kimball
  1060. via tomwhite)
  1061. MAPREDUCE-735. Fixes a problem in the KeyFieldHelper to do with
  1062. the end index for some inputs (Amar Kamat via ddas)
  1063. MAPREDUCE-682. Removes reservations on tasktrackers which are
  1064. blacklisted. (Sreekanth Ramakrishnan via yhemanth)
  1065. MAPREDUCE-743. Fixes a problem to do with progress reporting
  1066. in the map phase. (Ravi Gummadi via ddas)
  1067. MAPREDUCE-765. Eliminate the deprecated warnings introduced by H-5438.
  1068. (He Yongqiang via szetszwo)
  1069. MAPREDUCE-383. Fix a bug in Pipes combiner due to bytes count not
  1070. getting reset after the spill. (Christian Kunz via sharad)
  1071. MAPREDUCE-809. Fix job-summary logs to correctly record status of FAILED
  1072. and KILLED jobs. (acmurthy)
  1073. MAPREDUCE-792. Fix unchecked warnings in DBInputFormat. (Aaron Kimball
  1074. via szetszwo)
  1075. MAPREDUCE-760. Fix a timing issue in TestNodeRefresh. (Amar Kamat via
  1076. sharad)
  1077. MAPREDUCE-40. Keep memory management backwards compatible for job
  1078. configuration parameters and limits. (Rahul Kumar Singh via yhemanth)
  1079. MAPREDUCE-587. Fixes a OOM issue in TestStreamingExitStatus.
  1080. (Amar Kamat via ddas)
  1081. MAPREDUCE-408. Fixes an assertion problem in TestKillSubProcesses
  1082. (Ravi Gummadi via ddas)
  1083. MAPREDUCE-659. Fix gridmix2 compilation. (Giridharan Kesavan)
  1084. MAPREDUCE-796. Fixes a ClassCastException in an exception log in
  1085. MultiThreadedMapRunner. (Amar Kamat via ddas)
  1086. MAPREDUCE-808. Fixes a serialization problem in TypedBytes.
  1087. (Klaas Bosteels via ddas)
  1088. MAPREDUCE-845. Fix a findbugs heap size problem in build.xml and add
  1089. a new property findbugs.heap.size. (Lee Tucker via szetszwo)
  1090. MAPREDUCE-838. Fixes a problem in the way commit of task outputs
  1091. happens. The bug was that even if commit failed, the task would
  1092. be declared as successful. (Amareshwari Sriramadasu via ddas)
  1093. MAPREDUCE-813. Updates Streaming and M/R tutorial documents.
  1094. (Corinne Chandel via ddas)
  1095. MAPREDUCE-805. Fixes some deadlocks in the JobTracker due to the fact
  1096. the JobTracker lock hierarchy wasn't maintained in some JobInProgress
  1097. method calls. (Amar Kamat via ddas)
  1098. MAPREDUCE-799. Fixes so all of the MRUnit self-tests run.
  1099. (Aaron Kimball via johan)
  1100. MAPREDUCE-848. Fixes a problem to do with TestCapacityScheduler
  1101. failing (Amar Kamat via ddas)
  1102. MAPREDUCE-840. DBInputFormat leaves open transaction.
  1103. (Aaron Kimball via tomwhite)
  1104. MAPREDUCE-859. Adds Avro and its dependencies required by Hadoop
  1105. common. (Ravi Gummadi via sharad)
  1106. MAPREDUCE-867. Fix ivy conf to look for avro jar from maven repo.
  1107. (Giridharan Kesavan)
  1108. MAPREDUCE-877. Added avro as a dependency to contrib ivy settings.
  1109. (Tsz Wo (Nicholas) Sze via yhemanth)
  1110. MAPREDUCE-852. In build.xml, remove the Main-Class, which is incorrectly
  1111. set in tools, and rename the target "tools-jar" to "tools". (szetszwo)
  1112. MAPREDUCE-773. Sends progress reports for compressed gzip inputs in maps.
  1113. Fixes a native direct buffer leak in LineRecordReader classes.
  1114. (Hong Tang and ddas)
  1115. MAPREDUCE-832. Reduce number of warning messages printed when
  1116. deprecated memory variables are used. (Rahul Kumar Singh via yhemanth)
  1117. MAPREDUCE-745. Fixes a testcase problem to do with generation of JobTracker
  1118. IDs. (Amar Kamat via ddas)
  1119. MAPREDUCE-834. Enables memory management on tasktrackers when old
  1120. memory management parameters are used in configuration.
  1121. (Sreekanth Ramakrishnan via yhemanth)
  1122. MAPREDUCE-818. Fixes Counters#getGroup API. (Amareshwari Sriramadasu
  1123. via sharad)
  1124. MAPREDUCE-807. Handles the AccessControlException during the deletion of
  1125. mapred.system.dir in the JobTracker. The JobTracker will bail out if it
  1126. encounters such an exception. (Amar Kamat via ddas)
  1127. MAPREDUCE-430. Fix a bug related to task getting stuck in case of
  1128. OOM error. (Amar Kamat via ddas)
  1129. MAPREDUCE-871. Fix ownership of Job/Task local files to have correct
  1130. group ownership according to the egid of the tasktracker.
  1131. (Vinod Kumar Vavilapalli via yhemanth)
  1132. MAPREDUCE-911. Fix a bug in TestTaskFail related to speculative
  1133. execution. (Amareshwari Sriramadasu via sharad)
  1134. MAPREDUCE-687. Fix an assertion in TestMiniMRMapRedDebugScript.
  1135. (Amareshwari Sriramadasu via sharad)
  1136. MAPREDUCE-924. Fixes the TestPipes testcase to use Tool.
  1137. (Amareshwari Sriramadasu via sharad)
  1138. MAPREDUCE-903. Add Avro jar to eclipse classpath.
  1139. (Philip Zeyliger via tomwhite)
  1140. MAPREDUCE-943. Removes a testcase in TestNodeRefresh that doesn't make
  1141. sense in the new Job recovery model. (Amar Kamat via ddas)
  1142. MAPREDUCE-764. TypedBytesInput's readRaw() does not preserve custom type
  1143. codes. (Klaas Bosteels via tomwhite)
  1144. HADOOP-6243. Fixes a NullPointerException in handling deprecated keys.
  1145. (Sreekanth Ramakrishnan via yhemanth)
  1146. MAPREDUCE-968. NPE in distcp encountered when placing _logs directory on
  1147. S3FileSystem. (Aaron Kimball via tomwhite)
  1148. MAPREDUCE-826. harchive doesn't use ToolRunner / harchive returns 0 even
  1149. if the job fails with exception (koji Noguchi via mahadev)
  1150. MAPREDUCE-839. unit test TestMiniMRChildTask fails on mac os-x (hong tang
  1151. via mahadev)
  1152. MAPREDUCE-112. Add counters for reduce input, output records to the new API.
  1153. (Jothi Padmanabhan via cdouglas)
  1154. MAPREDUCE-648. Fix two distcp bugs: (1) it should not launch a job if all
  1155. src paths are directories, and (2) it does not skip copying when updating
  1156. a single file. (Ravi Gummadi via szetszwo)
  1157. MAPREDUCE-946. Fix a regression in LineRecordReader where the
  1158. maxBytesToConsume parameter is not set correctly. (cdouglas)
  1159. MAPREDUCE-977. Missing jackson jars from Eclipse template. (tomwhite)
  1160. MAPREDUCE-988. Fix a packaging issue in the contrib modules. (Hong Tang via
  1161. cdouglas)
  1162. MAPREDUCE-971. distcp does not always remove distcp.tmp.dir. (Aaron Kimball
  1163. via tomwhite)
  1164. MAPREDUCE-995. Fix a bug in JobHistory where tasks completing after the job
  1165. is closed cause a NPE. (Jothi Padmanabhan via cdouglas)
  1166. MAPREDUCE-953. Fix QueueManager to dump queue configuration in JSON format.
  1167. (V.V. Chaitanya Krishna via yhemanth)
  1168. MAPREDUCE-645. Prevent distcp from running a job when the destination is a
  1169. file, but the source is not. (Ravi Gummadi via cdouglas)
  1170. MAPREDUCE-1002. Flushed writer in JobQueueClient so queue information is
  1171. printed correctly. (V.V. Chaitanya Krishna via yhemanth)
  1172. MAPREDUCE-1003. Fix compilation problem in eclipse plugin when
  1173. eclipse.home is set. (Ravi Gummadi via yhemanth)
  1174. MAPREDUCE-941. Vaidya script fails on Solaris. (Chad Metcalf
  1175. via tomwhite)
  1176. MAPREDUCE-912. Add and standardize Apache license headers. (Chad Metcalf
  1177. via cdouglas)
  1178. MAPREDUCE-1022. Fix compilation of vertica testcases. (Vinod Kumar
  1179. Vavilapalli via acmurthy)
  1180. MAPREDUCE-1000. Handle corrupt history files in JobHistory.initDone().
  1181. (Jothi Padmanabhan via sharad)
  1182. MAPREDUCE-1028. Fixed number of slots occupied by cleanup tasks to one
  1183. irrespective of slot size for the job.
  1184. (Ravi Gummadi via yhemanth)
  1185. MAPREDUCE-964. Fixed start and finish times of TaskStatus to be
  1186. consistent, thereby fixing inconsistencies in metering tasks.
  1187. (Sreekanth Ramakrishnan via yhemanth)
  1188. MAPREDUCE-1076. Deprecate ClusterStatus and add javadoc in ClusterMetrics.
  1189. (Amareshwari Sriramadasu via sharad)
  1190. MAPREDUCE-979. Fixed JobConf APIs related to memory parameters to return
  1191. values of new configuration variables when deprecated variables are
  1192. disabled. (Sreekanth Ramakrishnan via yhemanth)
  1193. MAPREDUCE-1030. Modified scheduling algorithm to return a map and reduce
  1194. task per heartbeat in the capacity scheduler.
  1195. (Rahul Kumar Singh via yhemanth)
  1196. MAPREDUCE-1071. Use DataInputStream rather than FSDataInputStream in the
  1197. JobHistory EventReader. (Hong Tang via cdouglas)
  1198. MAPREDUCE-986. Fix Rumen to work with truncated task lines. (Dick King via
  1199. cdouglas)
  1200. MAPREDUCE-1029. Fix failing TestCopyFiles by restoring the unzipping of
  1201. HDFS webapps from the hdfs jar. (Aaron Kimball and Jothi Padmanabhan via
  1202. cdouglas)
  1203. MAPREDUCE-769. Make findbugs and javac warnings to zero.
  1204. (Amareshwari Sriramadasu via sharad)
  1205. MAPREDUCE-1104. Initialize RecoveryManager in JobTracker cstr called by
  1206. Mumak. (Hong Tang via cdouglas)
  1207. MAPREDUCE-1061. Add unit test validating byte specifications for gridmix
  1208. jobs. (cdouglas)
  1209. MAPREDUCE-1077. Fix Rumen so that truncated tasks do not mark the job as
  1210. successful. (Dick King via cdouglas)
  1211. MAPREDUCE-1041. Make TaskInProgress::taskStatuses map package-private.
  1212. (Jothi Padmanabhan via cdouglas)
  1213. MAPREDUCE-1070. Prevent a deadlock in the fair scheduler servlet.
  1214. (Todd Lipcon via cdouglas)
  1215. MAPREDUCE-1086. Setup Hadoop logging environment for tasks to point to
  1216. task related parameters. (Ravi Gummadi via yhemanth)
  1217. MAPREDUCE-1105. Remove max limit configuration in capacity scheduler in
  1218. favor of max capacity percentage thus allowing the limit to go over
  1219. queue capacity. (Rahul Kumar Singh via yhemanth)
  1220. MAPREDUCE-1016. Make the job history log format JSON. (cutting)
  1221. MAPREDUCE-1038. Weave Mumak aspects only if related files have changed.
  1222. (Aaron Kimball via cdouglas)
  1223. MAPREDUCE-1163. Remove unused, hard-coded paths from libhdfs. (Allen
  1224. Wittenauer via cdouglas)
  1225. MAPREDUCE-962. Fix a NullPointerException while killing task process
  1226. trees. (Ravi Gummadi via yhemanth)
  1227. MAPREDUCE-1177. Correct setup/cleanup inversion in
  1228. JobTracker::getTaskReports. (Vinod Kumar Vavilapalli via cdouglas)
  1229. MAPREDUCE-1178. Fix ClassCastException in MultipleInputs by adding
  1230. a DelegatingRecordReader. (Amareshwari Sriramadasu and Jay Booth
  1231. via sharad)
  1232. MAPREDUCE-1068. Fix streaming job to show proper message if file is
  1233. is not present. (Amareshwari Sriramadasu via sharad)
  1234. MAPREDUCE-1147. Add map output counters to new API. (Amar Kamat via
  1235. cdouglas)
  1236. MAPREDUCE-915. The debug scripts are run as the job user. (ddas)
  1237. MAPREDUCE-1007. Fix NPE in CapacityTaskScheduler.getJobs().
  1238. (V.V.Chaitanya Krishna via sharad)
  1239. MAPREDUCE-28. Refactor TestQueueManager and fix default ACLs.
  1240. (V.V.Chaitanya Krishna and Rahul K Singh via sharad)
  1241. MAPREDUCE-1182. Fix overflow in reduce causing allocations to exceed the
  1242. configured threshold. (cdouglas)
  1243. MAPREDUCE-1239. Fix contrib components build dependencies.
  1244. (Giridharan Kesavan and omalley)
  1245. MAPREDUCE-787. Fix JobSubmitter to honor user given symlink path.
  1246. (Amareshwari Sriramadasu via sharad)
  1247. MAPREDUCE-1249. Update config default value for socket read timeout to
  1248. match code default. (Amareshwari Sriramadasu via cdouglas)
  1249. MAPREDUCE-1161. Remove ineffective synchronization in NotificationTestCase.
  1250. (Owen O'Malley via cdouglas)
  1251. MAPREDUCE-1244. Fix eclipse-plugin's build dependencies. (gkesavan)
  1252. MAPREDUCE-1075. Fix JobTracker to not throw an NPE for a non-existent
  1253. queue. (V.V.Chaitanya Krishna via yhemanth)
  1254. MAPREDUCE-754. Fix NPE in expiry thread when a TT is lost. (Amar Kamat
  1255. via sharad)
  1256. MAPREDUCE-1074. Document Reducer mark/reset functionality. (Jothi
  1257. Padmanabhan via cdouglas)
  1258. MAPREDUCE-1267. Fix typo in mapred-default.xml. (Todd Lipcon via cdouglas)
  1259. MAPREDUCE-952. Remove inadvertently reintroduced Task.Counter enum. (Jothi
  1260. Padmanabhan via cdouglas)
  1261. MAPREDUCE-1230. Fix handling of null records in VerticaInputFormat. (Omer
  1262. Trajman via cdouglas)
  1263. MAPREDUCE-1171. Allow shuffle retries and read-error reporting to be
  1264. configurable. (Amareshwari Sriramadasu via acmurthy)
  1265. MAPREDUCE-879. Fix broken unit test TestTaskTrackerLocalization on MacOS.
  1266. (Sreekanth Ramakrishnan via yhemanth)
  1267. MAPREDUCE-1124. Fix imprecise byte counts in Gridmix. (cdouglas)
  1268. MAPREDUCE-1222. Add an option to exclude numeric IP addresses in topologies
  1269. processed by Mumak. (Hong Tang via cdouglas)
  1270. MAPREDUCE-1284. Fix fts_open() call in task-controller that was failing
  1271. LinuxTaskController unit tests. (Ravi Gummadi via yhemanth)
  1272. MAPREDUCE-1143. Fix running task counters to be updated correctly
  1273. when speculative attempts are running for a TIP.
  1274. (Rahul Kumar Singh via yhemanth)
  1275. MAPREDUCE-1241. Use a default queue configuration in JobTracker when
  1276. mapred-queues.xml is unavailable. (Todd Lipcon via cdouglas)
  1277. MAPREDUCE-1301. Fix set up of permission checking script used in
  1278. localization tests. (Amareshwari Sriramadasu via yhemanth)
  1279. MAPREDUCE-1286. Remove quoting from client opts in TaskRunner. (Yuri
  1280. Pradkin via cdouglas)
  1281. MAPREDUCE-1059. Use distcp.bytes.per.map when adding sync markers in
  1282. distcp. (Aaron Kimball via cdouglas)
  1283. MAPREDUCE-1009. Update forrest documentation describing hierarchical
  1284. queues. (Vinod Kumar Vavilapalli via yhemanth)
  1285. MAPREDUCE-1342. Fixed deadlock in global blacklisting of tasktrackers.
  1286. (Amareshwari Sriramadasu via acmurthy)
  1287. MAPREDUCE-1316. Fixes a memory leak of TaskInProgress instances in
  1288. the jobtracker. (Amar Kamat via yhemanth)
  1289. MAPREDUCE-1359. TypedBytes TestIO doesn't mkdir its test dir first.
  1290. (Anatoli Fomenko via cos)
  1291. MAPREDUCE-1314. Correct errant mapreduce.x.mapreduce.x replacements from
  1292. bulk change. (Amareshwari Sriramadasu via cdouglas)
  1293. MAPREDUCE-1365. Restore accidentally renamed test in
  1294. TestTaskTrackerBloacklisting. (Amareshwari Sriramadasu via cdouglas)
  1295. MAPREDUCE-1406. Fix spelling of JobContext.MAP_COMBINE_MIN_SPILLS.
  1296. (cdouglas)
  1297. MAPREDUCE-1369. JUnit tests should never depend on anything in conf
  1298. (Anatoli Fomenko via cos)
  1299. MAPREDUCE-1412. Fix timer granularity issue causing failures in
  1300. TestTaskTrackerBlacklisting. (cdouglas)
  1301. MAPREDUCE-1448. Respect --config option in Mumak script. (Hong Tang via
  1302. cdouglas)
  1303. MAPREDUCE-1251. c++ utils doesn't compile. (Eli Collins via tomwhite)
  1304. MAPREDUCE-1522. FileInputFormat may use the default FileSystem for the
  1305. input path. (Tsz Wo (Nicholas), SZE via cdouglas)
  1306. MAPREDUCE-1407. Update javadoc in mapreduce.{Mapper,Reducer} to match
  1307. actual usage. (Benoit Sigoure via cdouglas)
  1308. MAPREDUCE-1258. Fix fair scheduler event log not logging job info.
  1309. (matei)
  1310. MAPREDUCE-1089. Fix NPE in fair scheduler preemption when tasks are
  1311. scheduled but not running. (Todd Lipcon via matei)
  1312. MAPREDUCE-1014. Fix the libraries for common and hdfs. (omalley)
  1313. MAPREDUCE-1111. JT Jetty UI not working if we run mumak.sh
  1314. off packaged distribution directory. (hong tang via mahadev)
  1315. MAPREDUCE-1133. Eclipse .classpath template has outdated jar files and is
  1316. missing some new ones. (cos)
  1317. MAPREDUCE-1098. Fixed the distributed-cache to not do i/o while holding a
  1318. global lock. (Amareshwari Sriramadasu via acmurthy)
  1319. MAPREDUCE-1158. Fix JT running maps and running reduces metrics.
  1320. (sharad)
  1321. MAPREDUCE-1160. Reduce verbosity of log lines in some Map/Reduce classes
  1322. to avoid filling up jobtracker logs on a busy cluster.
  1323. (Ravi Gummadi and Hong Tang via yhemanth)
  1324. MAPREDUCE-1153. Fix tasktracker metrics when trackers are decommissioned.
  1325. (sharad)
  1326. MAPREDUCE-1128. Fix MRUnit to prohibit iterating over values twice. (Aaron
  1327. Kimball via cdouglas)
  1328. MAPREDUCE-665. Move libhdfs to HDFS subproject. (Eli Collins via dhruba)
  1329. MAPREDUCE-1196. Fix FileOutputCommitter to use the deprecated cleanupJob
  1330. api correctly. (acmurthy)
  1331. MAPREDUCE-1244. Fix eclipse-plugin's build dependencies. (gkesavan)
  1332. MAPREDUCE-1140. Fix DistributedCache to not decrement reference counts for
  1333. unreferenced files in error conditions.
  1334. (Amareshwari Sriramadasu via yhemanth)
  1335. MAPREDUCE-1245. Fix TestFairScheduler failures by instantiating lightweight
  1336. Jobtracker. (sharad)
  1337. MAPREDUCE-1260. Update Eclipse configuration to match changes to Ivy
  1338. configuration. (Edwin Chan via cos)
  1339. MAPREDUCE-1152. Distinguish between failed and killed tasks in
  1340. JobTrackerInstrumentation. (Sharad Agarwal via cdouglas)
  1341. MAPREDUCE-1285. In DistCp.deleteNonexisting(..), get class from the
  1342. parameter instead of using FileStatus.class. (Peter Romianowski via
  1343. szetszwo)
  1344. MAPREDUCE-1294. Build fails to pull latest hadoop-core-* artifacts (cos)
  1345. MAPREDUCE-1213. TaskTrackers restart is faster because it deletes
  1346. distributed cache directory asynchronously. (Zheng Shao via dhruba)
  1347. MAPREDUCE-1265. The task attempt error log prints the name of the
  1348. tasktracker machine. (Scott Chen via dhruba)
  1349. MAPREDUCE-1201. ProcfsBasedProcessTree collects CPU usage information.
  1350. (Scott Chen via dhruba)
  1351. MAPREDUCE-1326. fi tests don't use fi-site.xml (cos)
  1352. MAPREDUCE-1165. Replace non-portable function name with C99 equivalent.
  1353. (Allen Wittenauer via cdouglas)
  1354. MAPREDUCE-1331. Fixes a typo in a testcase (Devaraj Das)
  1355. MAPREDUCE-1293. AutoInputFormat doesn't work with non-default FileSystems.
  1356. (Andrew Hitchcock via tomwhite)
  1357. MAPREDUCE-1131. Using profilers other than hprof can cause JobClient to
  1358. report job failure. (Aaron Kimball via tomwhite)
  1359. MAPREDUCE-1155. Streaming tests swallow exceptions.
  1360. (Todd Lipcon via tomwhite)
  1361. MAPREDUCE-1212. Mapreduce contrib project ivy dependencies are not included
  1362. in binary target. (Aaron Kimball via tomwhite)
  1363. MAPREDUCE-1388. Move the HDFS RAID package from HDFS to MAPREDUCE.
  1364. (Eli Collins via dhruba)
  1365. MAPREDUCE-1322. Defines default value for staging directory to be user
  1366. based fixing a failing streaming test.
  1367. (Devaraj Das and Amareshwari Sriramadasu via yhemanth)
  1368. MAPREDUCE-899. Modified LinuxTaskController to check that task-controller
  1369. has right permissions and ownership before performing any actions.
  1370. (Amareshwari Sriramadasu via yhemanth)
  1371. MAPREDUCE-1443. DBInputFormat can leak connections.
  1372. (Aaron Kimball via tomwhite)
  1373. MAPREDUCE-1457. Fixes JobTracker to get the FileSystem object within
  1374. getStagingAreaDir within a privileged block. Fixes Child.java to use the
  1375. appropriate UGIs while getting the TaskUmbilicalProtocol proxy and
  1376. while executing the task. (Jakob Homan via ddas)
  1377. MAPREDUCE-1399. The archive command shows a null error message (nicholas
  1378. via mahadev)
  1379. MAPREDUCE-1305. Improve efficiency of distcp -delete. (Peter Romianowski
  1380. via cdouglas)
  1381. MAPREDUCE-1474. Update forrest documentation for Hadoop Archives. (Mahadev
  1382. Konar via cdouglas)
  1383. MAPREDUCE-1400. Use tr rather than sed to effect literal substitution in
  1384. the build script. (Allen Wittenauer via cdouglas)
  1385. MAPREDUCE-1358. Avoid false positives in OutputLogFilter. (Todd Lipcon via
  1386. cdouglas)
  1387. MAPREDUCE-1490. Fix a NullPointerException that could occur during
  1388. instantiation and initialization of the DistributedRaidFileSystem.
  1389. (Rodrigo Schmidt via dhruba)
  1390. MAPREDUCE-1476. Fix the M/R framework to not call commit for special
  1391. tasks like job setup/cleanup and task cleanup.
  1392. (Amareshwari Sriramadasu via yhemanth)
  1393. MAPREDUCE-1398. Fix TaskLauncher to stop waiting for slots on a TIP that
  1394. is killed / failed.
  1395. (Amareshwari Sriramadasu via yhemanth)
  1396. MAPREDUCE-1491. The parity files created by the RAID are combined
  1397. using Hadoop Archive Files (HAR). (Rodrigo Schmidt via dhruba)
  1398. MAPREDUCE-1378. URL encode link in jobhistory.jsp to avoid errors caused by
  1399. unescaped characters. (E. Sammer via cdouglas)
  1400. MAPREDUCE-1519. RaidNode fails to create new parity file
  1401. if an older version already exists. (Rodrigo Schmidt via dhruba)
  1402. MAPREDUCE-1537. Fixes a compilation problem in a testcase after commit
  1403. HDFS-984. (Jitendra Nath Pandey via ddas)
  1404. MAPREDUCE-1537. The patch makes the job client call the getDelegationToken
  1405. only when security is enabled. (Jitendra Nath Pandey via ddas)
  1406. MAPREDUCE-1510. RAID should regenerate parity files if they get deleted.
  1407. (Rodrigo Schmidt via dhruba)
  1408. MAPREDUCE-1421. Fix the LinuxTaskController tests failing on trunk after
  1409. the commit of MAPREDUCE-1385. (Amareshwari Sriramadasu via vinodkv)
  1410. MAPREDUCE-1520. Fix TestMiniMRLocalFS failure caused by regression in
  1411. getting user working dir. (Amareshwari Sriramadasu via cdouglas)
  1412. MAPREDUCE-1512. RAID uses HarFileSystem directly instead of
  1413. FileSystem.get (Rodrigo Schmidt via dhruba)
  1414. MAPREDUCE-1435. Fix symlink handling in task work directory when
  1415. cleaning up, essentially to avoid following links.
  1416. (Ravi Gummadi via yhemanth)
  1417. MAPREDUCE-1518. RaidNode does not run the deletion check on the
  1418. directory that stores the parity files. (Rodrigo Schmidt via dhruba)
  1419. MAPREDUCE-1573. TestStreamingAsDifferentUser fails if run as tt_user.
  1420. (Ravi Gummadi via vinodkv)
  1421. MAPREDUCE-927. Cleanup of task-logs should happen in TaskTracker instead
  1422. of the Child. (Amareshwari Sriramadasu via vinodkv)
  1423. MAPREDUCE-1578. Decouple HadoopArchives vesrion from HarFileSystem version.
  1424. (Rodrigo Schmidt via szetszwo)
  1425. MAPREDUCE-1422. Fix cleanup of localized job directory to work if files
  1426. with non-deletable permissions are created within it.
  1427. (Amar Kamat via yhemanth)
  1428. MAPREDUCE-1306. Randomize the arrival of heartbeat responses in Mumak.
  1429. (Tamas Sarlos via cdouglas)
  1430. MAPREDUCE-1579. archive: check and possibly replace the space charater
  1431. in source paths. (szetszwo)
  1432. MAPREDUCE-1536. DataDrivenDBInputFormat does not split date columns correctly.
  1433. (Aaron Kimball via enis)
  1434. MAPREDUCE-890. After HADOOP-4491, the user who started mapred system is
  1435. not able to run job. (Ravi Gummadi via vinodkv)
  1436. MAPREDUCE-1615. Fix compilation of TestSubmitJob. (cdouglas)
  1437. MAPREDUCE-1508. Protect against NPE in TestMultipleLevelCaching. (Aaron
  1438. Kimball via cdouglas)
  1439. MAPREDUCE-1497. Suppress spurious findbugs warning about IndexCache
  1440. synchronization. (Amareshwari Sriramadasu via cdouglas)
  1441. MAPREDUCE-1420. Fix TestTTResourceReporting failure. (Scott Chen via
  1442. cdouglas)
  1443. MAPREDUCE-1480. Correctly initialize child RecordReaders in
  1444. CombineFileInputFormat. (Aaron Kimball via cdouglas)
  1445. MAPREDUCE-1348. Fix block forensics packaging. (Tom White via cdouglas)
  1446. MAPREDUCE-1628. HarFileSystem shows incorrect replication numbers and
  1447. permissions. (szetszwo via mahadev)
  1448. MAPREDUCE-1602. Fix the error message for the case that src does not
  1449. exist. (szetszwo)
  1450. MAPREDUCE-1585. Create Hadoop Archives version 2 with filenames
  1451. URL-encoded (rodrigo via mahadev)
  1452. MAPREDUCE-1523. Sometimes rumen trace generator fails to extract the job
  1453. finish time. (dick king via mahadev)
  1454. MAPREDUCE-1635. ResourceEstimator does not work after MAPREDUCE-842.
  1455. (Amareshwari Sriramadasu via vinodkv)
  1456. MAPREDUCE-889. binary communication formats added to Streaming by
  1457. HADOOP-1722 should be documented. (Klaas Bosteels via tomwhite)
  1458. MAPREDUCE-1031. ant tar target doens't seem to compile tests in contrib
  1459. projects. (Aaron Kimball via tomwhite)
  1460. MAPREDUCE-1692. Removed unused testcase TestStreamedMerge.
  1461. (Sreekanth Ramakrishnan and Amareshwari Sriramadasu via yhemanth)
  1462. MAPREDUCE-1062. Fix ReliabilityTest to work with retired jobs. (Sreekanth
  1463. Ramakrishnan via cdouglas)
  1464. MAPREDUCE-1409. IOExceptions thrown from FIleOutputCommitter::abortTask
  1465. should cause the task to fail. (Amareshwari Sriramadasu via cdouglas)
  1466. MAPREDUCE-1695. Include capacity scheduler in findbugs and javadoc-dev
  1467. targets and also fix existing warnings. (Hong Tang via yhemanth)
  1468. MAPREDUCE-1494. Ensure TestJobDirCleanup verifies the correct paths.
  1469. (Amareshwari Sriramadasu via cdouglas)
  1470. MAPREDUCE-1622. Include missing slf4j dependencies. (cdouglas)
  1471. MAPREDUCE-1515. Accept java5.home from build.properties, not only from the
  1472. command line when building forrest docs. (Al Thompson via cdouglas)
  1473. MAPREDUCE-1618. Add missing javadoc to JobStatus::*JobAcls. (Amareshwari
  1474. Sriramadasu via cdouglas)
  1475. MAPREDUCE-1219. Remove job level metrics from jobtracker metrics to ease
  1476. undue load on jobtracker. (Sreekanth Ramakrishnan via sharad)
  1477. MAPREDUCE-1604. Add Forrest documentation for Job ACLs.
  1478. (Amareshwari Sriramadasu via yhemanth)
  1479. MAPREDUCE-1705. Archiving and Purging of HDFS parity files should
  1480. handle globbed policies accurately. (Rodrigo Schmidt via dhruba)
  1481. MAPREDUCE-1612. job conf file is not accessible from job history web page.
  1482. (Ravi Gummadi and Sreekanth Ramakrishnan via vinodkv)
  1483. MAPREDUCE-1397. NullPointerException observed during task failures.
  1484. (Amareshwari Sriramadasu via vinodkv)
  1485. MAPREDUCE-1728. Oracle timezone strings do not match Java.
  1486. (Aaron Kimball via tomwhite)
  1487. MAPREDUCE-1609. TaskTracker.localizeJob should not set permissions on
  1488. job log directory recursively. (Amareshwari Sriramadasu via vinodkv)
  1489. MAPREDUCE-1657. After task logs directory is deleted, tasklog servlet
  1490. displays wrong error message about job ACLs. (Ravi Gummadi via vinodkv)
  1491. MAPREDUCE-1727. TestJobACLs fails after HADOOP-6686. (Ravi Gummadi via vinodkv)
  1492. MAPREDUCE-1611. Refresh nodes and refresh queues doesnt work with service
  1493. authorization enabled. (Amar Kamat via vinodkv)
  1494. MAPREDUCE-1276. Correct flaws in the shuffle related to connection setup
  1495. and failure attribution. (Amareshwari Sriramadasu via cdouglas)
  1496. MAPREDUCE-1372. ConcurrentModificationException in JobInProgress.
  1497. (Dick King and Amareshwari Sriramadasu via tomwhite)
  1498. MAPREDUCE-118. Fix Job.getJobID(). (Amareshwari Sriramadasu via sharad)
  1499. MAPREDUCE-913. TaskRunner crashes with NPE resulting in held up slots,
  1500. UNINITIALIZED tasks and hung TaskTracker. (Amareshwari Sriramadasu and
  1501. Sreekanth Ramakrishnan via vinodkv)
  1502. MAPREDUCE-1725. Fix MapReduce API incompatibilities between 0.20 and 0.21.
  1503. (tomwhite)
  1504. MAPREDUCE-1606. TestJobACLs may timeout as there are no slots for launching
  1505. JOB_CLEANUP task. (Ravi Gummadi via vinodkv)
  1506. MAPREDUCE-1765. Correct streaming documentation for StreamXmlRecordReader.
  1507. (Corinne Chandel via amareshwari)
  1508. MAPREDUCE-1880. Fix BigDecimal.divide(..) in the pi example. (szetszwo)
  1509. MAPREDUCE-1885. Revert FileSystem create method that takes CreateFlags
  1510. (MapReduce part of HADOOP-6826). (Ravi Gummadi via tomwhite)
  1511. MAPREDUCE-1870. Harmonize MapReduce JAR library versions with Common and
  1512. HDFS. (tomwhite)
  1513. MAPREDUCE-1791. Remote cluster control functionality needs JavaDocs
  1514. improvement (Konstantin Boudnik)
  1515. MAPREDUCE-1942. 'compile-fault-inject' should never be called directly.
  1516. (Konstantin Boudnik)
  1517. MAPREDUCE-1876. Fixes TaskAttemptStartedEvent to correctly log event type
  1518. for all task types. (Amar Kamat via amareshwari)
  1519. MAPREDUCE-1926. MapReduce distribution is missing build-utils.xml.
  1520. (tomwhite)
  1521. MAPREDUCE-2012. Some contrib tests fail in branch 0.21 and trunk.
  1522. (Amareshwari Sriramadasu via tomwhite)
  1523. MAPREDUCE-1980. Fixes TaskAttemptUnsuccessfulCompletionEvent and
  1524. TaskAttemptFinishedEvent to correctly log event type for all task types.
  1525. (Amar Kamat via amareshwari)
  1526. MAPREDUCE-1856. Extract a subset of tests for smoke (DOA) validation (cos)