CHANGES.txt 196 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036103710381039104010411042104310441045104610471048104910501051105210531054105510561057105810591060106110621063106410651066106710681069107010711072107310741075107610771078107910801081108210831084108510861087108810891090109110921093109410951096109710981099110011011102110311041105110611071108110911101111111211131114111511161117111811191120112111221123112411251126112711281129113011311132113311341135113611371138113911401141114211431144114511461147114811491150115111521153115411551156115711581159116011611162116311641165116611671168116911701171117211731174117511761177117811791180118111821183118411851186118711881189119011911192119311941195119611971198119912001201120212031204120512061207120812091210121112121213121412151216121712181219122012211222122312241225122612271228122912301231123212331234123512361237123812391240124112421243124412451246124712481249125012511252125312541255125612571258125912601261126212631264126512661267126812691270127112721273127412751276127712781279128012811282128312841285128612871288128912901291129212931294129512961297129812991300130113021303130413051306130713081309131013111312131313141315131613171318131913201321132213231324132513261327132813291330133113321333133413351336133713381339134013411342134313441345134613471348134913501351135213531354135513561357135813591360136113621363136413651366136713681369137013711372137313741375137613771378137913801381138213831384138513861387138813891390139113921393139413951396139713981399140014011402140314041405140614071408140914101411141214131414141514161417141814191420142114221423142414251426142714281429143014311432143314341435143614371438143914401441144214431444144514461447144814491450145114521453145414551456145714581459146014611462146314641465146614671468146914701471147214731474147514761477147814791480148114821483148414851486148714881489149014911492149314941495149614971498149915001501150215031504150515061507150815091510151115121513151415151516151715181519152015211522152315241525152615271528152915301531153215331534153515361537153815391540154115421543154415451546154715481549155015511552155315541555155615571558155915601561156215631564156515661567156815691570157115721573157415751576157715781579158015811582158315841585158615871588158915901591159215931594159515961597159815991600160116021603160416051606160716081609161016111612161316141615161616171618161916201621162216231624162516261627162816291630163116321633163416351636163716381639164016411642164316441645164616471648164916501651165216531654165516561657165816591660166116621663166416651666166716681669167016711672167316741675167616771678167916801681168216831684168516861687168816891690169116921693169416951696169716981699170017011702170317041705170617071708170917101711171217131714171517161717171817191720172117221723172417251726172717281729173017311732173317341735173617371738173917401741174217431744174517461747174817491750175117521753175417551756175717581759176017611762176317641765176617671768176917701771177217731774177517761777177817791780178117821783178417851786178717881789179017911792179317941795179617971798179918001801180218031804180518061807180818091810181118121813181418151816181718181819182018211822182318241825182618271828182918301831183218331834183518361837183818391840184118421843184418451846184718481849185018511852185318541855185618571858185918601861186218631864186518661867186818691870187118721873187418751876187718781879188018811882188318841885188618871888188918901891189218931894189518961897189818991900190119021903190419051906190719081909191019111912191319141915191619171918191919201921192219231924192519261927192819291930193119321933193419351936193719381939194019411942194319441945194619471948194919501951195219531954195519561957195819591960196119621963196419651966196719681969197019711972197319741975197619771978197919801981198219831984198519861987198819891990199119921993199419951996199719981999200020012002200320042005200620072008200920102011201220132014201520162017201820192020202120222023202420252026202720282029203020312032203320342035203620372038203920402041204220432044204520462047204820492050205120522053205420552056205720582059206020612062206320642065206620672068206920702071207220732074207520762077207820792080208120822083208420852086208720882089209020912092209320942095209620972098209921002101210221032104210521062107210821092110211121122113211421152116211721182119212021212122212321242125212621272128212921302131213221332134213521362137213821392140214121422143214421452146214721482149215021512152215321542155215621572158215921602161216221632164216521662167216821692170217121722173217421752176217721782179218021812182218321842185218621872188218921902191219221932194219521962197219821992200220122022203220422052206220722082209221022112212221322142215221622172218221922202221222222232224222522262227222822292230223122322233223422352236223722382239224022412242224322442245224622472248224922502251225222532254225522562257225822592260226122622263226422652266226722682269227022712272227322742275227622772278227922802281228222832284228522862287228822892290229122922293229422952296229722982299230023012302230323042305230623072308230923102311231223132314231523162317231823192320232123222323232423252326232723282329233023312332233323342335233623372338233923402341234223432344234523462347234823492350235123522353235423552356235723582359236023612362236323642365236623672368236923702371237223732374237523762377237823792380238123822383238423852386238723882389239023912392239323942395239623972398239924002401240224032404240524062407240824092410241124122413241424152416241724182419242024212422242324242425242624272428242924302431243224332434243524362437243824392440244124422443244424452446244724482449245024512452245324542455245624572458245924602461246224632464246524662467246824692470247124722473247424752476247724782479248024812482248324842485248624872488248924902491249224932494249524962497249824992500250125022503250425052506250725082509251025112512251325142515251625172518251925202521252225232524252525262527252825292530253125322533253425352536253725382539254025412542254325442545254625472548254925502551255225532554255525562557255825592560256125622563256425652566256725682569257025712572257325742575257625772578257925802581258225832584258525862587258825892590259125922593259425952596259725982599260026012602260326042605260626072608260926102611261226132614261526162617261826192620262126222623262426252626262726282629263026312632263326342635263626372638263926402641264226432644264526462647264826492650265126522653265426552656265726582659266026612662266326642665266626672668266926702671267226732674267526762677267826792680268126822683268426852686268726882689269026912692269326942695269626972698269927002701270227032704270527062707270827092710271127122713271427152716271727182719272027212722272327242725272627272728272927302731273227332734273527362737273827392740274127422743274427452746274727482749275027512752275327542755275627572758275927602761276227632764276527662767276827692770277127722773277427752776277727782779278027812782278327842785278627872788278927902791279227932794279527962797279827992800280128022803280428052806280728082809281028112812281328142815281628172818281928202821282228232824282528262827282828292830283128322833283428352836283728382839284028412842284328442845284628472848284928502851285228532854285528562857285828592860286128622863286428652866286728682869287028712872287328742875287628772878287928802881288228832884288528862887288828892890289128922893289428952896289728982899290029012902290329042905290629072908290929102911291229132914291529162917291829192920292129222923292429252926292729282929293029312932293329342935293629372938293929402941294229432944294529462947294829492950295129522953295429552956295729582959296029612962296329642965296629672968296929702971297229732974297529762977297829792980298129822983298429852986298729882989299029912992299329942995299629972998299930003001300230033004300530063007300830093010301130123013301430153016301730183019302030213022302330243025302630273028302930303031303230333034303530363037303830393040304130423043304430453046304730483049305030513052305330543055305630573058305930603061306230633064306530663067306830693070307130723073307430753076307730783079308030813082308330843085308630873088308930903091309230933094309530963097309830993100310131023103310431053106310731083109311031113112311331143115311631173118311931203121312231233124312531263127312831293130313131323133313431353136313731383139314031413142314331443145314631473148314931503151315231533154315531563157315831593160316131623163316431653166316731683169317031713172317331743175317631773178317931803181318231833184318531863187318831893190319131923193319431953196319731983199320032013202320332043205320632073208320932103211321232133214321532163217321832193220322132223223322432253226322732283229323032313232323332343235323632373238323932403241324232433244324532463247324832493250325132523253325432553256325732583259326032613262326332643265326632673268326932703271327232733274327532763277327832793280328132823283328432853286328732883289329032913292329332943295329632973298329933003301330233033304330533063307330833093310331133123313331433153316331733183319332033213322332333243325332633273328332933303331333233333334333533363337333833393340334133423343334433453346334733483349335033513352335333543355335633573358335933603361336233633364336533663367336833693370337133723373337433753376337733783379338033813382338333843385338633873388338933903391339233933394339533963397339833993400340134023403340434053406340734083409341034113412341334143415341634173418341934203421342234233424342534263427342834293430343134323433343434353436343734383439344034413442344334443445344634473448344934503451345234533454345534563457345834593460346134623463346434653466346734683469347034713472347334743475347634773478347934803481348234833484348534863487348834893490349134923493349434953496349734983499350035013502350335043505350635073508350935103511351235133514351535163517351835193520352135223523352435253526352735283529353035313532353335343535353635373538353935403541354235433544354535463547354835493550355135523553355435553556355735583559356035613562356335643565356635673568356935703571357235733574357535763577357835793580358135823583358435853586358735883589359035913592359335943595359635973598359936003601360236033604360536063607360836093610361136123613361436153616361736183619362036213622362336243625362636273628362936303631363236333634363536363637363836393640364136423643364436453646364736483649365036513652365336543655365636573658365936603661366236633664366536663667366836693670367136723673367436753676367736783679368036813682368336843685368636873688368936903691369236933694369536963697369836993700370137023703370437053706370737083709371037113712371337143715371637173718371937203721372237233724372537263727372837293730373137323733373437353736373737383739374037413742374337443745374637473748374937503751375237533754375537563757375837593760376137623763376437653766376737683769377037713772377337743775377637773778377937803781378237833784378537863787378837893790379137923793379437953796379737983799380038013802380338043805380638073808380938103811381238133814381538163817381838193820382138223823382438253826382738283829383038313832383338343835383638373838383938403841384238433844384538463847384838493850385138523853385438553856385738583859386038613862386338643865386638673868386938703871387238733874387538763877387838793880388138823883388438853886388738883889389038913892389338943895389638973898389939003901390239033904390539063907390839093910391139123913391439153916391739183919392039213922392339243925392639273928392939303931393239333934393539363937393839393940394139423943394439453946394739483949395039513952395339543955395639573958395939603961396239633964396539663967396839693970397139723973397439753976397739783979398039813982398339843985398639873988398939903991399239933994399539963997399839994000400140024003400440054006400740084009401040114012401340144015401640174018401940204021402240234024402540264027402840294030403140324033403440354036403740384039404040414042404340444045404640474048404940504051405240534054405540564057405840594060406140624063406440654066406740684069407040714072407340744075407640774078407940804081408240834084408540864087408840894090409140924093409440954096409740984099410041014102410341044105410641074108410941104111411241134114411541164117411841194120412141224123412441254126412741284129413041314132413341344135413641374138413941404141414241434144414541464147414841494150415141524153415441554156415741584159416041614162416341644165416641674168416941704171417241734174417541764177417841794180418141824183418441854186418741884189419041914192419341944195419641974198419942004201420242034204420542064207420842094210421142124213421442154216421742184219422042214222422342244225422642274228422942304231423242334234423542364237423842394240424142424243424442454246424742484249425042514252425342544255425642574258425942604261426242634264426542664267426842694270427142724273427442754276427742784279428042814282428342844285428642874288428942904291429242934294429542964297429842994300430143024303430443054306430743084309431043114312431343144315431643174318431943204321432243234324432543264327432843294330433143324333433443354336433743384339434043414342434343444345434643474348434943504351435243534354435543564357435843594360436143624363436443654366436743684369437043714372437343744375437643774378437943804381438243834384438543864387438843894390439143924393439443954396439743984399440044014402440344044405440644074408440944104411441244134414441544164417441844194420442144224423442444254426442744284429443044314432443344344435443644374438443944404441444244434444444544464447444844494450445144524453445444554456445744584459446044614462446344644465446644674468446944704471447244734474447544764477447844794480448144824483448444854486448744884489449044914492449344944495449644974498449945004501450245034504450545064507450845094510451145124513451445154516451745184519452045214522452345244525452645274528452945304531453245334534453545364537453845394540454145424543454445454546454745484549455045514552455345544555455645574558455945604561456245634564456545664567456845694570457145724573457445754576457745784579458045814582458345844585458645874588458945904591459245934594459545964597459845994600460146024603460446054606460746084609461046114612461346144615461646174618461946204621462246234624462546264627462846294630463146324633463446354636463746384639464046414642464346444645464646474648464946504651465246534654465546564657465846594660466146624663466446654666466746684669467046714672467346744675467646774678467946804681468246834684468546864687468846894690469146924693469446954696469746984699470047014702470347044705470647074708470947104711471247134714471547164717471847194720472147224723472447254726472747284729473047314732473347344735473647374738473947404741474247434744474547464747474847494750475147524753475447554756475747584759476047614762476347644765476647674768476947704771477247734774477547764777477847794780478147824783478447854786478747884789479047914792479347944795479647974798479948004801480248034804480548064807480848094810481148124813481448154816481748184819482048214822482348244825482648274828482948304831483248334834483548364837483848394840484148424843484448454846484748484849485048514852485348544855485648574858485948604861486248634864486548664867486848694870487148724873487448754876487748784879488048814882488348844885488648874888488948904891489248934894489548964897489848994900490149024903490449054906490749084909491049114912491349144915491649174918491949204921492249234924492549264927492849294930493149324933493449354936493749384939494049414942494349444945494649474948494949504951495249534954495549564957495849594960496149624963496449654966496749684969497049714972497349744975497649774978497949804981498249834984498549864987498849894990499149924993499449954996499749984999500050015002500350045005500650075008500950105011501250135014501550165017501850195020502150225023502450255026502750285029503050315032503350345035503650375038503950405041504250435044504550465047504850495050505150525053505450555056505750585059506050615062506350645065506650675068506950705071507250735074507550765077507850795080508150825083508450855086508750885089509050915092509350945095509650975098509951005101510251035104510551065107510851095110511151125113511451155116511751185119512051215122512351245125512651275128512951305131513251335134513551365137513851395140514151425143514451455146514751485149515051515152515351545155515651575158515951605161516251635164516551665167
  1. Hadoop MapReduce Change Log
  2. Release 0.23.3 - UNRELEASED
  3. INCOMPATIBLE CHANGES
  4. MAPREDUCE-4072. User set java.library.path seems to overwrite default
  5. creating problems native lib loading (Anupam Seth via bobby)
  6. MAPREDUCE-3812. Lower default allocation sizes, fix allocation
  7. configurations and document them (Harsh J via bobby)
  8. NEW FEATURES
  9. IMPROVEMENTS
  10. MAPREDUCE-4059. The history server should have a separate pluggable
  11. storage/query interface. (Robert Evans via tgraves)
  12. MAPREDUCE-3942. Randomize master key generation for
  13. ApplicationTokenSecretManager and roll it every so often. (Vinod Kumar
  14. Vavilapalli via sseth)
  15. MAPREDUCE-4151. RM scheduler web page should filter apps to those that
  16. are relevant to scheduling (Jason Lowe via tgraves)
  17. MAPREDUCE-4134. Remove references of mapred.child.ulimit etc. since they
  18. are not being used any more (Ravi Prakash via bobby)
  19. MAPREDUCE-3972. Fix locking and exception issues in JobHistory server.
  20. (Robert Joseph Evans via sseth)
  21. MAPREDUCE-4161. create sockets consistently (Daryn Sharp via bobby)
  22. MAPREDUCE-4079. Allow MR AppMaster to limit ephemeral port range.
  23. (bobby via tgraves)
  24. MAPREDUCE-4210. Expose listener address for WebApp (Daryn Sharp via bobby)
  25. MAPREDUCE-4162. Correctly set token service (Daryn Sharp via bobby)
  26. OPTIMIZATIONS
  27. MAPREDUCE-3850. Avoid redundant calls for tokens in TokenCache (Daryn
  28. Sharp via bobby)
  29. BUG FIXES
  30. MAPREDUCE-4092. commitJob Exception does not fail job (Jon Eagles via
  31. bobby)
  32. MAPREDUCE-4089. Hung Tasks never time out. (Robert Evans via tgraves)
  33. MAPREDUCE-4024. RM webservices can't query on finalStatus (Tom Graves
  34. via bobby)
  35. MAPREDUCE-4060. Multiple SLF4J binding warning (Jason Lowe via bobby)
  36. MAPREDUCE-3983. TestTTResourceReporting can fail, and should just be
  37. deleted (Ravi Prakash via bobby)
  38. MAPREDUCE-4012 Hadoop Job setup error leaves no useful info to users
  39. (when LinuxTaskController is used). (tgraves)
  40. MAPREDUCE-4062. AM Launcher thread can hang forever (tgraves via bobby)
  41. MAPREDUCE-3988. mapreduce.job.local.dir doesn't point to a single directory
  42. on a node. (Eric Payne via bobby)
  43. MAPREDUCE-3999. Tracking link gives an error if the AppMaster hasn't
  44. started yet (Ravi Prakash via bobby)
  45. MAPREDUCE-4020. Web services returns incorrect JSON for deep queue tree
  46. (Anupam Seth via tgraves)
  47. MAPREDUCE-3672. Killed maps shouldn't be counted towards
  48. JobCounter.NUM_FAILED_MAPS. (Anupam Seth via tgraves)
  49. MAPREDUCE-3682 Tracker URL says AM tasks run on localhost.
  50. (Ravi Prakash via tgraves)
  51. MAPREDUCE-3082. Archive command take wrong path for input file with current
  52. directory (John George via bobby)
  53. MAPREDUCE-3650. testGetTokensForHftpFS() fails (Ravi Prakash via bobby)
  54. MAPREDUCE-3621. TestDBJob and TestDataDrivenDBInputFormat ant tests fail
  55. (Ravi Prakash via tgraves)
  56. MAPREDUCE-4073. CS assigns multiple off-switch containers when using
  57. multi-level-queues (Siddharth Seth via bobby)
  58. MAPREDUCE-4051. Remove the empty hadoop-mapreduce-project/assembly/all.xml
  59. file (Ravi Prakash via bobby)
  60. MAPREDUCE-4117. mapred job -status throws NullPointerException (Devaraj K
  61. via bobby)
  62. MAPREDUCE-4099. ApplicationMaster may fail to remove staging directory
  63. (Jason Lowe via bobby)
  64. MAPREDUCE-4017. Add jobname to jobsummary log (tgraves and Koji Noguchi
  65. via bobby)
  66. MAPREDUCE-4040. History links should use hostname rather than IP address.
  67. (Bhallamudi Venkata Siva Kamesh via sseth)
  68. MAPREDUCE-4099 amendment. ApplicationMaster will remove staging directory
  69. after the history service is stopped. (Jason Lowe via sseth)
  70. MAPREDUCE-3932. Fix the TaskAttempt state machine to handle
  71. CONTIANER_LAUNCHED and CONTIANER_LAUNCH_FAILED events in additional
  72. states. (Robert Joseph Evans via sseth)
  73. MAPREDUCE-4140. mapreduce classes incorrectly importing
  74. "clover.org.apache.*" classes. (Patrick Hunt via tomwhite)
  75. MAPREDUCE-4050. For tasks without assigned containers, changes the node
  76. text on the UI to N/A instead of a link to null. (Bhallamudi Venkata Siva
  77. Kamesh via sseth)
  78. MAPREDUCE-4128. AM Recovery expects all attempts of a completed task to
  79. also be completed. (Bikas Saha via bobby)
  80. MAPREDUCE-4144. Fix a NPE in the ResourceManager when handling node
  81. updates. (Jason Lowe via sseth)
  82. MAPREDUCE-4156. ant build fails compiling JobInProgress (tgraves)
  83. MAPREDUCE-4160. some mrv1 ant tests fail with timeout - due to 4156
  84. (tgraves)
  85. MAPREDUCE-4074. Client continuously retries to RM When RM goes down
  86. before launching Application Master (xieguiming via tgraves)
  87. MAPREDUCE-4159. Job is running in Uber mode after setting
  88. "mapreduce.job.ubertask.maxreduces" to zero (Devaraj K via bobby)
  89. MAPREDUCE-4165. Committing is misspelled as commiting in task logs
  90. (John Eagles via bobby)
  91. MAPREDUCE-4129. Lots of unneeded counters log messages (Ahmed Radwan via
  92. bobby)
  93. MAPREDUCE-3947. yarn.app.mapreduce.am.resource.mb not documented
  94. (Devaraj K via bobby)
  95. MAPREDUCE-4190. Improve web UI for task attempts userlog link (Tom Graves
  96. via bobby)
  97. MAPREDUCE-4133. MR over viewfs is broken (John George via bobby)
  98. MAPREDUCE-4194. ConcurrentModificationError in DirectoryCollection
  99. (Jonathan Eagles via bobby)
  100. MAPREDUCE-3613. web service calls header contains 2 content types
  101. (tgraves)
  102. MAPREDUCE-4169. Container Logs appear in unsorted order (Jonathan Eagles
  103. via bobby)
  104. MAPREDUCE-4189. TestContainerManagerSecurity is failing (Devaraj K via
  105. bobby)
  106. MAPREDUCE-4209. junit dependency in hadoop-mapreduce-client is missing
  107. scope test (Radim Kolar via bobby)
  108. MAPREDUCE-4206. Sorting by Last Health-Update on the RM nodes page sorts
  109. does not work correctly (Jonathon Eagles via tgraves)
  110. MAPREDUCE-4212. TestJobClientGetJob sometimes fails
  111. (Daryn Sharp via tgraves)
  112. MAPREDUCE-4211. Error conditions (missing appid, appid not found) are
  113. masked in the RM app page (Jonathan Eagles via bobby)
  114. MAPREDUCE-4163. consistently set the bind address (Daryn Sharp via bobby)
  115. MAPREDUCE-4048. NullPointerException exception while accessing the
  116. Application Master UI (Devaraj K via bobby)
  117. MAPREDUCE-4220. RM apps page starttime/endtime sorts are incorrect
  118. (Jonathan Eagles via bobby)
  119. MAPREDUCE-4226. ConcurrentModificationException in FileSystemCounterGroup.
  120. (tomwhite)
  121. MAPREDUCE-4215. RM app page shows 500 error on appid parse error
  122. (Jonathon Eagles via tgraves)
  123. MAPREDUCE-4237. TestNodeStatusUpdater can fail if localhost has a domain
  124. associated with it (bobby)
  125. Release 0.23.2 - UNRELEASED
  126. INCOMPATIBLE CHANGES
  127. NEW FEATURES
  128. IMPROVEMENTS
  129. MAPREDUCE-3849. Change TokenCache's reading of the binary token file
  130. (Daryn Sharp via bobby)
  131. MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's
  132. environmental variables in TestMiniMRChildTask. (Tom White via vinodkv)
  133. MAPREDUCE-3877 Add a test to formalise the current state transitions
  134. of the yarn lifecycle. (stevel)
  135. MAPREDUCE-3866. Fixed the bin/yarn script to not print the command line
  136. unnecessarily. (vinodkv)
  137. MAPREDUCE-3730. Modified RM to allow restarted NMs to be able to join the
  138. cluster without waiting for expiry. (Jason Lowe via vinodkv)
  139. MAPREDUCE-2793. Corrected AppIDs, JobIDs, TaskAttemptIDs to be of correct
  140. format on the web pages. (Bikas Saha via vinodkv)
  141. MAPREDUCE-3614. Fixed MR AM to close history file quickly and send a correct
  142. final state to the RM when it is killed. (Ravi Prakash via vinodkv)
  143. MAPREDUCE-3497. Added docs for YARN CLI. (tgraves via acmurthy)
  144. MAPREDUCE-3954. Added new envs to separate heap size for different daemons
  145. started via bin scripts. (Robert Joseph Evans via vinodkv)
  146. MAPREDUCE-4025. AM can crash if task attempt reports bogus progress value
  147. (Jason Lowe via bobby)
  148. MAPREDUCE-4034. Unable to view task logs on history server with
  149. mapreduce.job.acl-view-job=* (Jason Lowe and Siddarth Seth via bobby)
  150. MAPREDUCE-4043. Secret keys set in Credentials are not seen by tasks
  151. (Jason Lowe via bobby)
  152. OPTIMIZATIONS
  153. MAPREDUCE-3901. Modified JobHistory records in YARN to lazily load job and
  154. task reports so as to improve UI response times. (Siddarth Seth via vinodkv)
  155. MAPREDUCE-2855. Passing a cached class-loader to ResourceBundle creator to
  156. minimize counter names lookup time. (Siddarth Seth via vinodkv)
  157. MAPREDUCE-3944. Change the history jobs/ webservice to return partial job
  158. info for a significant performance improvement. (Robert Joseph Evans via
  159. sseth)
  160. BUG FIXES
  161. MAPREDUCE-3918 proc_historyserver no longer in command line arguments for
  162. HistoryServer (Jon Eagles via bobby)
  163. MAPREDUCE-3862. Nodemanager can appear to hang on shutdown due to lingering
  164. DeletionService threads (Jason Lowe via bobby)
  165. MAPREDUCE-3680. FifoScheduler web service rest API can print out invalid
  166. JSON. (B Anil Kumar via tgraves)
  167. MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas
  168. Graves via mahadev)
  169. MAPREDUCE-3864. Fix cluster setup docs for correct SecondaryNameNode
  170. HTTPS parameters. (todd)
  171. MAPREDUCE-3583. Change pid to String and stime to BigInteger in order to
  172. avoid NumberFormatException caused by overflow. (Zhihong Yu via szetszwo)
  173. MAPREDUCE-3634. Fixed all daemons to crash instead of hanging around when
  174. their EventHandlers get exceptions. (vinodkv)
  175. MAPREDUCE-3798. Fixed failing TestJobCleanup.testCusomCleanup() and moved it
  176. to the maven build. (Ravi Prakash via vinodkv)
  177. MAPREDUCE-3884. PWD should be first in the classpath of MR tasks (tucu)
  178. MAPREDUCE-3878. Null user on filtered jobhistory job page (Jonathon Eagles
  179. via tgraves)
  180. MAPREDUCE-3738. MM can hang during shutdown if AppLogAggregatorImpl thread
  181. dies unexpectedly (Jason Lowe via sseth)
  182. MAPREDUCE-3904 Job history produced with mapreduce.cluster.acls.enabled
  183. false can not be viewed with mapreduce.cluster.acls.enabled true
  184. (Jonathon Eagles via tgraves)
  185. MAPREDUCE-3910. Fixed a bug in CapacityScheduler LeafQueue which was causing
  186. app-submission to fail. (John George via vinodkv)
  187. MAPREDUCE-3686. Fixed two bugs in Counters because of which web app displays
  188. zero counter values for framework counters. (Bhallamudi Venkata Siva Kamesh
  189. via vinodkv)
  190. MAPREDUCE-3913. RM application webpage is unresponsive after 2000 jobs
  191. (Jason Lowe via tgraves)
  192. MAPREDUCE-3922. Fixed build to not compile 32bit container-executor binary
  193. by default on all platforms. (Hitesh Shah via vinodkv)
  194. MAPREDUCE-3790. Broken pipe on streaming job can lead to truncated output for
  195. a successful job (Jason Lowe via bobby)
  196. MAPREDUCE-3816. capacity scheduler web ui bar graphs for used capacity wrong
  197. (tgraves via bobby)
  198. MAPREDUCE-3930. Fixed an NPE while accessing the AM page/webservice for a
  199. task attempt without an assigned container. (Robert Joseph Evans via
  200. sseth)
  201. MAPREDUCE-3931. Changed PB implementation of LocalResource to take locks
  202. so that race conditions don't fail tasks by inadvertantly changing the
  203. timestamps. (Siddarth Seth via vinodkv)
  204. MAPREDUCE-3687. If AM dies before it returns new tracking URL, proxy
  205. redirects to http://N/A/ and doesn't return error code (Ravi Prakash via
  206. bobby)
  207. MAPREDUCE-3920. Revise yarn default port number selection
  208. (Dave Thompson via tgraves)
  209. MAPREDUCE-3903. Add support for mapreduce admin users. (Thomas Graves via
  210. sseth)
  211. MAPREDUCE-3706. Fix circular redirect error in job-attempts page. (bobby
  212. via acmurthy)
  213. MAPREDUCE-3896. Add user information to the delegation token issued by the
  214. history server. (Vinod Kumar Vavilapalli via sseth)
  215. MAPREDUCE-3792. Fix "bin/mapred job -list" to display all jobs instead of
  216. only the jobs owned by the user. (Jason Lowe via vinodkv)
  217. MAPREDUCE-3929. Fixed output of 'bin/mapred queue -showacl' command to
  218. clarify ACLs for users. (John George via acmurthy)
  219. MAPREDUCE-3960. Fix web-proxy to forward request to AM with configured
  220. hostname or IP. (tgraves via acmurthy)
  221. MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by
  222. using capacity and not max-capacity since we are already scaling it by
  223. userLimitFactor. (Eric Payne via acmurthy)
  224. MAPREDUCE-3009. Fixed node link on JobHistory webapp. (chackaravarthy via
  225. vinodkv)
  226. MAPREDUCE-3964. ResourceManager does not have JVM metrics (Jason Lowe via
  227. bobby)
  228. MAPREDUCE-3034. Ensure NodeManager reboots itself on direction from
  229. ResourceManager. (Devaraj K & Eric Payne via acmurthy)
  230. MAPREDUCE-3976. TestRMContainerAllocator failing (Jason Lowe via bobby)
  231. MAPREDUCE-3961. Map/ReduceSlotMillis computation incorrect (Siddharth Seth
  232. via bobby)
  233. MAPREDUCE-3977. LogAggregationService leaks log aggregator objects
  234. (Jason Lowe via bobby)
  235. MAPREDUCE-3975. Default value not set for Configuration parameter
  236. mapreduce.job.local.dir (Eric Payne via bobby)
  237. MAPREDUCE-3982. Fixed FileOutputCommitter to not err out for an 'empty-job'
  238. whose tasks don't write any outputs. (Robert Joseph Evans via vinodkv)
  239. MAPREDUCE-4005. Fixes broken AM container logs URL on ResourceManager
  240. Application Info page. (Jason Lowe via sseth)
  241. MAPREDUCE-4006. History server container log web UI sometimes combines
  242. stderr/stdout/syslog contents together (Siddharth Seth via tgraves)
  243. MAPREDUCE-4061. RM only has 1 AM launcher thread (tgraves via bobby)
  244. Release 0.23.1 - 2012-02-17
  245. INCOMPATIBLE CHANGES
  246. NEW FEATURES
  247. MAPREDUCE-778. Rumen Anonymizer. (Amar Kamat and Chris Douglas via amarrk)
  248. MAPREDUCE-3121. NodeManager should handle disk-failures (Ravi Gummadi via mahadev)
  249. MAPREDUCE-2863. Support web services for YARN and MR components. (Thomas
  250. Graves via vinodkv)
  251. MAPREDUCE-3251. Network ACLs can prevent some clients to talk to MR ApplicationMaster.
  252. (Anupam Seth via mahadev)
  253. MAPREDUCE-778. Rumen Anonymizer. (Amar Kamat and Chris Douglas via amarrk)
  254. IMPROVEMENTS
  255. MAPREDUCE-3481. [Gridmix] Improve Gridmix STRESS mode. (amarrk)
  256. MAPREDUCE-3597. [Rumen] Rumen should provide APIs to access all the
  257. job-history related information.
  258. MAPREDUCE-3375. [Gridmix] Memory Emulation system tests.
  259. (Vinay Thota via amarrk)
  260. MAPREDUCE-3840. JobEndNotifier doesn't use the proxyToUse during connecting
  261. (Ravi Prakash via bobby)
  262. MAPREDUCE-3736. Variable substitution depth too large for fs.default.name
  263. causes jobs to fail (ahmed via tucu).
  264. MAPREDUCE-2733. [Gridmix] Gridmix3 cpu emulation system tests.
  265. (Vinay Thota via amarrk)
  266. MAPREDUCE-3297. Moved log related components into yarn-common so that
  267. HistoryServer and clients can use them without depending on the
  268. yarn-server-nodemanager module. (Siddharth Seth via vinodkv)
  269. MAPREDUCE-3336. Replaced guice internal.Preconditions api usage with the
  270. public Preconditions API. (Thomas Graves via vinodkv)
  271. MAPREDUCE-3280. Removed the unnecessary job user-name configuration in
  272. mapred-site.xml. (vinodkv)
  273. MAPREDUCE-3370. Fixed MiniMRYarnCluster and related tests to not use
  274. a hard-coded path for the mr-app jar. (Ahmed Radwan via vinodkv)
  275. MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves
  276. via mahadev)
  277. MAPREDUCE-3341. Enhance logging of initalized queue limit values.
  278. (Anupam Seth via mahadev)
  279. MAPREDUCE-3243. Invalid tracking URL for streaming jobs (Jonathan Eagles
  280. via mahadev)
  281. MAPREDUCE-3331. Improvement to single node cluster setup documentation for
  282. 0.23 (Anupam Seth via mahadev)
  283. MAPREDUCE-3102. Changed NodeManager to fail fast when LinuxContainerExecutor
  284. has wrong configuration or permissions. (Hitesh Shah via vinodkv)
  285. MAPREDUCE-3415. improve MiniMRYarnCluster & DistributedShell JAR
  286. resolution. (tucu)
  287. MAPREDUCE-3169. Create a new MiniMRCluster equivalent which only provides
  288. client APIs cross MR1 and MR2. (Ahmed via tucu)
  289. MAPREDUCE-3373. Hadoop scripts unconditionally source
  290. "$bin"/../libexec/hadoop-config.sh. (Bruno Mahé via tomwhite)
  291. MAPREDUCE-3372. HADOOP_PREFIX cannot be overridden.
  292. (Bruno Mahé via tomwhite)
  293. MAPREDUCE-3411. Performance Upgrade for jQuery (Jonathan Eagles via
  294. mahadev)
  295. MAPREDUCE-3371. Review and improve the yarn-api javadocs. (Ravi Prakash
  296. via mahadev)
  297. MAPREDUCE-3238. Small cleanup in SchedulerApp. (Todd Lipcon via mahadev)
  298. MAPREDUCE-3413. RM web ui applications not sorted in any order by default.
  299. (Jonathan Eagles via mahadev)
  300. MAPREDUCE-3045. Fixed UI filters to not filter on hidden title-numeric
  301. sort fields. (Jonathan Eagles via sseth)
  302. MAPREDUCE-3448. TestCombineOutputCollector javac unchecked warning on mocked
  303. generics (Jonathan Eagles via mahadev)
  304. MAPREDUCE-3169 amendment. Deprecate MiniMRCluster. (Ahmed Radwan via
  305. sseth)
  306. MAPREDUCE-3369. Migrate MR1 tests to run on MR2 using the new interfaces
  307. introduced in MAPREDUCE-3169. (Ahmed Radwan via tomwhite)
  308. MAPREDUCE-3518. mapred queue -info <queue> -showJobs throws NPE.
  309. (Jonathan Eagles via mahadev)
  310. MAPREDUCE-3391. Making a trivial change to correct a log message in
  311. DistributedShell app's AM. (Subroto Sanyal via vinodkv)
  312. MAPREDUCE-3547. Added a bunch of unit tests for the the RM/NM webservices.
  313. (Thomas Graves via acmurthy)
  314. MAPREDUCE-3610. Remove use of the 'dfs.block.size' config for default block
  315. size fetching. Use FS#getDefaultBlocksize instead. (Sho Shimauchi via harsh)
  316. MAPREDUCE-3478. Cannot build against ZooKeeper 3.4.0. (Tom White via mahadev)
  317. MAPREDUCE-3528. Fixed TaskHeartBeatHandler to use a new configuration
  318. for the thread loop interval separate from task-timeout configuration
  319. property. (Siddharth Seth via vinodkv)
  320. MAPREDUCE-3312. Modified MR AM to not send a stop-container request for
  321. a container that isn't launched at all. (Robert Joseph Evans via vinodkv)
  322. MAPREDUCE-3382. Enhanced MR AM to use a proxy to ping the job-end
  323. notification URL. (Ravi Prakash via vinodkv)
  324. MAPREDUCE-3299. Added AMInfo table to the MR AM job pages to list all the
  325. job-attempts when AM restarts and recovers. (Jonathan Eagles via vinodkv)
  326. MAPREDUCE-3251. Network ACLs can prevent some clients to talk to MR AM.
  327. Improved the earlier patch to not to JobHistoryServer repeatedly.
  328. (Anupam Seth via vinodkv)
  329. MAPREDUCE-3553. Add support for data returned when exceptions thrown from web
  330. service apis to be in either xml or in JSON. (Thomas Graves via mahadev)
  331. MAPREDUCE-3641. Making CapacityScheduler more conservative so as to
  332. assign only one off-switch container in a single scheduling
  333. iteration. (Arun C Murthy via vinodkv)
  334. MAPREDUCE-3692. yarn-resourcemanager out and log files can get big. (eli)
  335. MAPREDUCE-3710. Improved FileInputFormat to return better locality for the
  336. last split. (Siddarth Seth via vinodkv)
  337. MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)
  338. MAPREDUCE-3737. The Web Application Proxy's is not documented very well.
  339. (Robert Evans via mahadev)
  340. MAPREDUCE-3699. Increased RPC handlers for all YARN servers to reasonable
  341. values for working at scale. (Hitesh Shah via vinodkv)
  342. MAPREDUCE-3693. Added mapreduce.admin.user.env to mapred-default.xml.
  343. (Roman Shapshonik via acmurthy)
  344. MAPREDUCE-3732. Modified CapacityScheduler to use only users with pending
  345. requests for computing user-limits. (Arun C Murthy via vinodkv)
  346. MAPREDUCE-3679. AM logs and others should not automatically refresh after every 1
  347. second. (Vinod KV via mahadev)
  348. MAPREDUCE-3754. Modified RM UI to filter applications based on state of the
  349. applications. (vinodkv)
  350. MAPREDUCE-3774. Moved yarn-default.xml to hadoop-yarn-common from
  351. hadoop-server-common. (Mahadev Konar via vinodkv)
  352. MAPREDUCE-3771. Un-deprecated the old mapred apis, port of MAPREDUCE-1735.
  353. (acmurthy)
  354. MAPREDUCE-3784. Fixed CapacityScheduler so that maxActiveApplications and
  355. maxActiveApplicationsPerUser per queue are not too low for small
  356. clusters. (Arun C Murthy via vinodkv)
  357. OPTIMIZATIONS
  358. MAPREDUCE-3567. Extraneous JobConf objects in AM heap. (Vinod Kumar
  359. Vavilapalli via sseth)
  360. MAPREDUCE-3399. Modifying ContainerLocalizer to send a heartbeat to NM
  361. immediately after downloading a resource instead of always waiting for a
  362. second. (Siddarth Seth via vinodkv)
  363. MAPREDUCE-3568. Optimized Job's progress calculations in MR AM. (vinodkv)
  364. MAPREDUCE-3569. TaskAttemptListener holds a global lock for all
  365. task-updates. (Vinod Kumar Vavilapalli via sseth)
  366. MAPREDUCE-3511. Removed a multitude of cloned/duplicate counters in the AM
  367. thereby reducing the AM heap size and preventing full GCs. (vinodkv)
  368. MAPREDUCE-3618. Fixed TaskHeartbeatHandler to not hold a global lock for all
  369. task-updates. (Siddarth Seth via vinodkv)
  370. MAPREDUCE-3512. Batching JobHistory flushing to DFS so that we don't flush
  371. for every event slowing down AM. (Siddarth Seth via vinodkv)
  372. MAPREDUCE-3718. Change default AM heartbeat interval to 1 second. (Hitesh
  373. Shah via sseth)
  374. MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes
  375. on the webapps. (Bhallamudi Venkata Siva Kamesh and Jason Lowe via vinodkv)
  376. MAPREDUCE-3756. Made single shuffle limit configurable. (Hitesh Shah via
  377. acmurthy)
  378. MAPREDUCE-3811. Made jobclient-to-AM retries configurable. (sseth via
  379. acmurthy)
  380. BUG FIXES
  381. MAPREDUCE-2784. [Gridmix] Bug fixes in ExecutionSummarizer and
  382. ResourceUsageMatcher. (amarrk)
  383. MAPREDUCE-3194. "mapred mradmin" command is broken in mrv2
  384. (Jason Lowe via bobby)
  385. MAPREDUCE-3462. Fix Gridmix JUnit testcase failures.
  386. (Ravi Prakash and Ravi Gummadi via amarrk)
  387. MAPREDUCE-2950. [Rumen] Fixed TestUserResolve. (Ravi Gummadi via amarrk)
  388. MAPREDUCE-3412. Fix 'ant docs'. (amarrk)
  389. MAPREDUCE-3346 [Rumen] LoggedTaskAttempt#getHostName() returns null.
  390. (amarrk)
  391. MAPREDUCE-3221. Reenabled the previously ignored test in TestSubmitJob
  392. and fixed bugs in it. (Devaraj K via vinodkv)
  393. MAPREDUCE-3215. Reenabled and fixed bugs in the failing test
  394. TestNoJobSetupCleanup. (Hitesh Shah via vinodkv)
  395. MAPREDUCE-3219. Reenabled and fixed bugs in the failing test
  396. TestDelegationToken. (Hitesh Shah via vinodkv)
  397. MAPREDUCE-3217. Reenabled and fixed bugs in the failing ant test
  398. TestAuditLogger. (Devaraj K via vinodkv)
  399. MAPREDUCE-3291. App fail to launch due to delegation token not
  400. found in cache (Robert Evans via mahadev)
  401. MAPREDUCE-3344. o.a.h.mapreduce.Reducer since 0.21 blindly casts to
  402. ReduceContext.ValueIterator. (Brock Noland via tomwhite)
  403. MAPREDUCE-3342. Fixed JobHistoryServer to also show the job's queue
  404. name. (Jonathan Eagles via vinodkv)
  405. MAPREDUCE-3345. Fixed a race condition in ResourceManager that was causing
  406. TestContainerManagerSecurity to fail sometimes. (Hitesh Shah via vinodkv)
  407. MAPREDUCE-3368. Fixed test compilation. (Hitesh Shah via vinodkv)
  408. MAPREDUCE-3333. Fixed bugs in ContainerLauncher of MR AppMaster due to
  409. which per-container connections to NodeManager were lingering long enough
  410. to hit the ulimits on number of processes. (vinodkv)
  411. MAPREDUCE-3392. Fixed Cluster's getDelegationToken's API to return null
  412. when there isn't a supported token. (John George via vinodkv)
  413. MAPREDUCE-3379. Fixed LocalResourceTracker in NodeManager to remove deleted
  414. cache entries correctly. (Siddharth Seth via vinodkv)
  415. MAPREDUCE-3324. Not All HttpServer tools links (stacks,logs,config,metrics) are
  416. accessible through all UI servers (Jonathan Eagles via mahadev)
  417. MAPREDUCE-3355. Fixed MR AM's ContainerLauncher to handle node-command
  418. timeouts correctly. (vinodkv)
  419. MAPREDUCE-3407. Fixed pom files to refer to the correct MR app-jar needed
  420. by the integration tests. (Hitesh Shah via vinodkv)
  421. MAPREDUCE-3437. Fix examples pom to refer to the correct 0.23 snapshot
  422. version. (Jonathan Eagles via todd)
  423. MAPREDUCE-3434. Nightly build broken (Hitesh Shah via mahadev)
  424. MAPREDUCE-3447. mapreduce examples not working (mahadev)
  425. MAPREDUCE-3444. trunk/0.23 builds broken (Hitesh Shah via mahadev)
  426. MAPREDUCE-3454. [Gridmix] TestDistCacheEmulation is broken (Hitesh Shah
  427. via mahadev)
  428. MAPREDUCE-3408. yarn-daemon.sh unconditionnaly sets yarn.root.logger
  429. (Bruno Mahe via mahadev)
  430. MAPREDUCE-3329. Fixed CapacityScheduler to ensure maximum-capacity cannot
  431. be lesser than capacity for any queue. (acmurthy)
  432. MAPREDUCE-3464. mapreduce jsp pages missing DOCTYPE. (Dave Vronay via mattf)
  433. MAPREDUCE-3265. Removed debug logs during job submission to LOG.debug to
  434. cut down noise. (acmurthy)
  435. MAPREDUCE-3468. Changed ant based infrastructure to use 0.23.1 version.
  436. (sseth via acmurthy)
  437. MAPREDUCE-3433. Finding counters by legacy group name returns empty
  438. counters. (tomwhite)
  439. MAPREDUCE-3450. NM port info no longer available in JobHistory.
  440. (Siddharth Seth via mahadev)
  441. MAPREDUCE-3477. Hadoop site documentation cannot be built anymore.
  442. (jeagles via tucu)
  443. MAPREDUCE-3488. Streaming jobs are failing because the main class
  444. isnt set in the pom files. (mahadev)
  445. MAPREDUCE-3463. Second AM fails to recover properly when first AM is killed with
  446. java.lang.IllegalArgumentException causing lost job. (Siddharth Seth via mahadev)
  447. MAPREDUCE-3452. fifoscheduler web ui page always shows 0% used for the queue.
  448. (Jonathan Eagles via mahadev)
  449. MAPREDUCE-3443. JobClient and Job should function in the context of the
  450. UGI which created them. (Mahadev Konar via sseth)
  451. MAPREDUCE-3460. MR AM can hang if containers are allocated on a node
  452. blacklisted by the AM. (Hitesh Shah and Robert Joseph Evans via sseth)
  453. MAPREDUCE-3453. RM web ui application details page shows RM cluster about
  454. information. (Jonathan Eagles via sseth)
  455. MAPREDUCE-3479. JobClient#getJob cannot find local jobs. (tomwhite)
  456. MAPREDUCE-3500. MRJobConfig creates an LD_LIBRARY_PATH using the platform ARCH. (tucu)
  457. MAPREDUCE-3456. $HADOOP_PREFIX/bin/yarn should set defaults for
  458. $HADOOP_*_HOME (Eric Payne via mahadev)
  459. MAPREDUCE-3458. Fix findbugs warnings in hadoop-examples. (Devaraj K
  460. via mahadev)
  461. MAPREDUCE-3485. DISKS_FAILED -101 error code should be defined in same location as
  462. ABORTED_CONTAINER_EXIT_STATUS. (Ravi Gummadi via mahadev)
  463. MAPREDUCE-3389. MRApps loads the 'mrapp-generated-classpath' file with
  464. classpath from the build machine. (tucu)
  465. MAPREDUCE-3496. Fixed client to print queue acls in consistent order.
  466. (Jonathan Eagles via acmurthy)
  467. MAPREDUCE-3147. Handle leaf queues with the same name properly.
  468. (Ravi Prakash via mahadev)
  469. MAPREDUCE-3327. RM web ui scheduler link doesn't show correct max value
  470. for queues (Anupam Seth via mahadev)
  471. MAPREDUCE-3513. Capacity Scheduler web UI has a spelling mistake for Memory.
  472. (chackaravarthy via mahadev)
  473. MAPREDUCE-3519. Fixed a deadlock in NodeManager LocalDirectories's handling
  474. service. (Ravi Gummadi via vinodkv)
  475. MAPREDUCE-3527. Fix minor API incompatibilities between 1.0 and 0.23.
  476. (tomwhite)
  477. MAPREDUCE-3328. mapred queue -list output inconsistent and missing child
  478. queues. (Ravi Prakash via mahadev)
  479. MAPREDUCE-3510. Capacity Scheduler inherited ACLs not displayed by mapred queue
  480. -showacls (Jonathan Eagles via mahadev)
  481. MAPREDUCE-3537. Fix race condition in DefaultContainerExecutor which led
  482. to container localization occuring in wrong directories. (acmurthy)
  483. MAPREDUCE-3542. Support "FileSystemCounter" legacy counter group name for
  484. compatibility. (tomwhite)
  485. MAPREDUCE-3426. Fixed MR AM in uber mode to write map intermediate outputs
  486. in the correct directory to work properly in secure mode. (Hitesh Shah via
  487. vinodkv)
  488. MAPREDUCE-3544. gridmix build is broken, requires hadoop-archives to be
  489. added as ivy dependency. (tucu)
  490. MAPREDUCE-3557. MR1 test fail to compile because of missing hadoop-archives
  491. dependency. (tucu)
  492. MAPREDUCE-3541. Fix broken TestJobQueueClient test. (Ravi Prakash via
  493. mahadev)
  494. MAPREDUCE-3398. Fixed log aggregation to work correctly in secure mode.
  495. (Siddharth Seth via vinodkv)
  496. MAPREDUCE-3530. Fixed an NPE occuring during scheduling in the
  497. ResourceManager. (Arun C Murthy via vinodkv)
  498. MAPREDUCE-3484. Fixed JobEndNotifier to not get interrupted before completing
  499. all its retries. (Ravi Prakash via vinodkv)
  500. MAPREDUCE-3531. Fixed a race in ContainerTokenSecretManager. (Robert Joseph
  501. Evans via sseth)
  502. MAPREDUCE-3560. TestRMNodeTransitions is failing on trunk.
  503. (Siddharth Seth via mahadev)
  504. MAPREDUCE-3487. Fixed JobHistory web-UI to display links to single task's
  505. counters' page. (Jason Lowe via vinodkv)
  506. MAPREDUCE-3564. Fixed failures in TestStagingCleanup and TestJobEndNotifier
  507. tests. (Siddharth Seth via vinodkv)
  508. MAPREDUCE-3422. Counter display names are not being picked up. (Jonathan
  509. Eagles via sseth)
  510. MAPREDUCE-3366. Mapreduce component should use consistent directory structure
  511. layout as HDFS/common (Eric Yang via mahadev)
  512. MAPREDUCE-3387. Fixed AM's tracking URL to always go through the proxy, even
  513. before the job started, so that it works properly with oozie throughout
  514. the job execution. (Robert Joseph Evans via vinodkv)
  515. MAPREDUCE-3579. ConverterUtils shouldn't include a port in a path from a url
  516. without a port. (atm via harsh)
  517. MAPREDUCE-3563. Fixed LocalJobRunner to work correctly with new mapreduce
  518. apis. (acmurthy)
  519. MAPREDUCE-3376. Fixed Task to ensure it passes reporter to combiners using
  520. old MR api. (Subroto Sanyal via acmurthy)
  521. MAPREDUCE-3339. Fixed MR AM to stop considering node blacklisting after the
  522. number of nodes blacklisted crosses a threshold. (Siddharth Seth via vinodkv)
  523. MAPREDUCE-3588. Fixed bin/yarn which was broken by MAPREDUCE-3366 so that
  524. yarn daemons can start. (Arun C Murthy via vinodkv)
  525. MAPREDUCE-3349. Log rack-name in JobHistory for unsuccessful tasks. (Amar
  526. Kamat and Devaraj K via sseth)
  527. MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations
  528. thereby solving race conditions in MR AM shutdown. (vinodkv)
  529. MAPREDUCE-3604. Fixed streaming to use new mapreduce.framework.name to
  530. check for local mode. (acmurthy)
  531. MAPREDUCE-3521. Fixed streaming to ensure it doesn't silently ignore
  532. unknown arguments. (Robert Evans via acmurthy)
  533. MAPREDUCE-3522. Ensure queues inherit ACLs from parent if they aren't
  534. explicitly specified. (Jonathan Eagles via acmurthy)
  535. MAPREDUCE-3608. Fixed compile issue with MAPREDUCE-3522. (mahadev via
  536. acmurthy)
  537. MAPREDUCE-3490. Fixed MapReduce AM to count failed maps also towards Reduce
  538. ramp up. (Sharad Agarwal and Arun C Murthy via vinodkv)
  539. MAPREDUCE-3529. TokenCache does not cache viewfs credentials correctly
  540. (sseth)
  541. MAPREDUCE-3595. Add missing TestCounters#testCounterValue test from branch
  542. 1 to 0.23 (Tom White via sseth)
  543. MAPREDUCE-3566. Fixed MR AM to construct CLC only once across all tasks.
  544. (vinodkv via acmurthy)
  545. MAPREDUCE-3572. Moved AM event dispatcher to a separate thread for
  546. performance reasons. (vinodkv via acmurthy)
  547. MAPREDUCE-3615. Fix some ant test failures. (Thomas Graves via sseth)
  548. MAPREDUCE-1744. DistributedCache creates its own FileSytem instance when
  549. adding a file/archive to the path. (Dick King via tucu)
  550. MAPREDUCE-3326. Added detailed information about queue's to the
  551. CapacityScheduler web-ui. (Jason Lowe via acmurthy)
  552. MAPREDUCE-3548. Added more unit tests for MR AM & JHS web-services.
  553. (Thomas Graves via acmurthy)
  554. MAPREDUCE-3617. Removed wrong default value for
  555. yarn.resourcemanager.principal and yarn.nodemanager.principal. (Jonathan
  556. Eagles via acmurthy)
  557. MAPREDUCE-3624. Remove unnecessary dependency on JDK's tools.jar. (mahadev
  558. via acmurthy)
  559. MAPREDUCE-3616. Thread pool for launching containers in MR AM not
  560. expanding as expected. (vinodkv via sseth)
  561. MAPREDUCE-3639. Fixed TokenCache to work with absent FileSystem canonical
  562. service-names. (Siddharth Seth via vinodkv)
  563. MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos
  564. authenticated. (mahadev)
  565. MAPREDUCE-3648. TestJobConf failing. (Thomas Graves via mahadev)
  566. MAPREDUCE-3651. TestQueueManagerRefresh fails. (Thomas Graves via mahadev)
  567. MAPREDUCE-3645. TestJobHistory fails. (Thomas Graves via mahadev)
  568. MAPREDUCE-3652. org.apache.hadoop.mapred.TestWebUIAuthorization.testWebUIAuthorization
  569. fails. (Thomas Graves via mahadev)
  570. MAPREDUCE-3625. CapacityScheduler web-ui display of queue's used capacity is broken.
  571. (Jason Lowe via mahadev)
  572. MAPREDUCE-3596. Fix scheduler to handle cleaned up containers, which NMs
  573. may subsequently report as running. (Vinod Kumar Vavilapalli via sseth)
  574. MAPREDUCE-3656. Fixed a race condition in MR AM which is failing the sort
  575. benchmark consistently. (Siddarth Seth via vinodkv)
  576. MAPREDUCE-3532. Modified NM to report correct http address when an ephemeral
  577. web port is configured. (Bhallamudi Venkata Siva Kamesh via vinodkv)
  578. MAPREDUCE-3404. Corrected MR AM to honor speculative configuration and enable
  579. speculating either maps or reduces. (Eric Payne via vinodkv)
  580. MAPREDUCE-3664. Federation Documentation has incorrect configuration example.
  581. (Brandon Li via jitendra)
  582. MAPREDUCE-3649. Job End notification gives an error on calling back.
  583. (Ravi Prakash via mahadev)
  584. MAPREDUCE-3657. State machine visualize build fails. (Jason Lowe
  585. via mahadev)
  586. MAPREDUCE-2450. Fixed a corner case with interrupted communication threads
  587. leading to a long timeout in Task. (Rajesh Balamohan via acmurthy)
  588. MAPREDUCE-3669. Allow clients to talk to MR HistoryServer using both
  589. delegation tokens and kerberos. (mahadev via acmurthy)
  590. MAPREDUCE-3684. LocalDistributedCacheManager does not shut down its thread
  591. pool (tomwhite)
  592. MAPREDUCE-3582. Move successfully passing MR1 tests to MR2 maven tree.
  593. (ahmed via tucu)
  594. MAPREDUCE-3698. Client cannot talk to the history server in secure mode.
  595. (mahadev)
  596. MAPREDUCE-3689. RM web UI doesn't handle newline in job name.
  597. (Thomas Graves via mahadev)
  598. MAPREDUCE-3701. Delete HadoopYarnRPC from 0.23 branch.
  599. (mahadev)
  600. MAPREDUCE-3549. write api documentation for web service apis for RM, NM,
  601. mapreduce app master, and job history server (Thomas Graves via mahadev)
  602. MAPREDUCE-3705. ant build fails on 0.23 branch. (Thomas Graves via
  603. mahadev)
  604. MAPREDUCE-3691. webservices add support to compress response.
  605. (Thomas Graves via mahadev)
  606. MAPREDUCE-3702. internal server error trying access application master
  607. via proxy with filter enabled (Thomas Graves via mahadev)
  608. MAPREDUCE-3646. Remove redundant URL info from "mapred job" output.
  609. (Jonathan Eagles via mahadev)
  610. MAPREDUCE-3681. Fixed computation of queue's usedCapacity. (acmurthy)
  611. MAPREDUCE-3505. yarn APPLICATION_CLASSPATH needs to be overridable.
  612. (ahmed via tucu)
  613. MAPREDUCE-3714. Fixed EventFetcher and Fetcher threads to shut-down properly
  614. so that reducers don't hang in corner cases. (vinodkv)
  615. MAPREDUCE-3712. The mapreduce tar does not contain the hadoop-mapreduce-client-
  616. jobclient-tests.jar. (mahadev)
  617. MAPREDUCE-3717. JobClient test jar has missing files to run all the test programs.
  618. (mahadev)
  619. MAPREDUCE-3630. Fixes a NullPointer exception while running TeraGen - if a
  620. map is asked to generate 0 records. (Mahadev Konar via sseth)
  621. MAPREDUCE-3683. Fixed maxCapacity of queues to be product of parent
  622. maxCapacities. (acmurthy)
  623. MAPREDUCE-3713. Fixed the way head-room is allocated to applications by
  624. CapacityScheduler so that it deducts current-usage per user and not
  625. per-application. (Arun C Murthy via vinodkv)
  626. MAPREDUCE-3721. Fixed a race in shuffle which caused reduces to hang.
  627. (sseth via acmurthy)
  628. MAPREDUCE-3733. Add Apache License Header to hadoop-distcp/pom.xml.
  629. (mahadev)
  630. MAPREDUCE-3735. Add distcp jar to the distribution (tar).
  631. (mahadev)
  632. MAPREDUCE-3720. Changed bin/mapred job -list to not print job-specific
  633. information not available at RM. (vinodkv via acmurthy)
  634. MAPREDUCE-3742. "yarn logs" command fails with ClassNotFoundException.
  635. (Jason Lowe via mahadev)
  636. MAPREDUCE-3703. ResourceManager should provide node lists in JMX output.
  637. (Eric Payne via mahadev)
  638. MAPREDUCE-3716. Fixing YARN+MR to allow MR jobs to be able to use
  639. java.io.File.createTempFile to create temporary files as part of their
  640. tasks. (Jonathan Eagles via vinodkv)
  641. MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug.
  642. (ramya via acmurthy)
  643. MAPREDUCE-3764. Fixed resource usage metrics for queues and users.
  644. (acmurthy)
  645. MAPREDUCE-3749. ConcurrentModificationException in counter groups.
  646. (tomwhite)
  647. MAPREDUCE-3762. Fixed default CapacityScheduler configs. (mahadev via
  648. acmurthy)
  649. MAPREDUCE-3499. New MiniMR does not setup proxyuser configuration
  650. correctly, thus tests using doAs do not work. (johnvijoe via tucu)
  651. MAPREDUCE-3696. MR job via oozie does not work on hadoop 23.
  652. (John George via mahadev)
  653. MAPREDUCE-3427. Fix streaming unit tests broken after mavenization.
  654. (Hitesh Shah via acmurthy)
  655. MAPREDUCE-3640. Allow AMRecovery to work with partial JobHistory files.
  656. (Arun C Murthy via sseth)
  657. MAPREDUCE-3752. Modified application limits to include queue max-capacities
  658. besides the usual user limits. (Arun C Murthy via vinodkv)
  659. MAPREDUCE-3744. Fix the yarn logs command line. Improve error messages for
  660. mapred job -logs. (Jason Lowe via sseth)
  661. MAPREDUCE-3780. Fixed a bug where applications killed before getting
  662. activated were not getting cleaned up properly. (Hitesh Shah via acmurthy)
  663. MAPREDUCE-3708. Metrics: Incorrect Apps Submitted Count (Bhallamudi via
  664. mahadev)
  665. MAPREDUCE-3727. jobtoken location property in jobconf refers to wrong
  666. jobtoken file (tucu)
  667. MAPREDUCE-3711. Fixed MR AM recovery so that only single selected task
  668. output is recovered and thus reduce the unnecessarily bloated recovery
  669. time. (Robert Joseph Evans via vinodkv)
  670. MAPREDUCE-3760. Changed active nodes list to not contain unhealthy nodes
  671. on the webUI and metrics. (vinodkv)
  672. MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and
  673. JobHistoryServer web-apps. (Jonathan Eagles via vinodkv)
  674. MAPREDUCE-3803. Fix broken build of raid contrib due to HDFS-2864.
  675. (Ravi Prakash via suresh)
  676. MAPREDUCE-3791. can't build site in hadoop-yarn-server-common.
  677. (mahadev)
  678. MAPREDUCE-3723. TestAMWebServicesJobs & TestHSWebServicesJobs
  679. incorrectly asserting tests (Bhallamudi Venkata Siva Kamesh
  680. via mahadev)
  681. MAPREDUCE-3795. "job -status" command line output is malformed.
  682. (vinodkv via mahadev)
  683. MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when
  684. there are a few unhealthy nodes (vinodkv via mahadev)
  685. MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname.
  686. (Hitesh Shah via mahadev)
  687. MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-
  688. allocation-mb setting (Hitesh Shah via mahadev)
  689. MAPREDUCE-3747. Initialize queue metrics upfront and added start/finish
  690. time to RM Web-UI. (acmurthy)
  691. MAPREDUCE-3814. Fixed MRV1 compilation. (Arun C Murthy via vinodkv)
  692. MAPREDUCE-3810. Performance tweaks - reduced logging in AM and defined
  693. hascode/equals for ResourceRequest & Priority. (vinodkv via acmurthy)
  694. MAPREDUCE-3813. Added a cache for resolved racks. (vinodkv via acmurthy)
  695. MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps
  696. but no reduces. (Robert Joseph Evans via vinodkv)
  697. MAPREDUCE-3804. yarn webapp interface vulnerable to cross scripting attacks
  698. (Dave Thompson via bobby)
  699. MAPREDUCE-3354. Changed scripts so that jobhistory server is started by
  700. bin/mapred instead of bin/yarn. (Jonathan Eagles via acmurthy)
  701. MAPREDUCE-3809. Ensure that there is no needless sleep in Task at the end
  702. of the task. (sseth via acmurthy)
  703. MAPREDUCE-3794. Support mapred.Task.Counter and mapred.JobInProgress.Counter
  704. enums for compatibility (Tom White via mahadev)
  705. MAPREDUCE-3697. Support binary compatibility for Counters after
  706. MAPREDUCE-901. (mahadev via acmurthy)
  707. MAPREDUCE-3817. Fixed bin/mapred to allow running of distcp and archive
  708. jobs. (Arpit Gupta via acmurthy)
  709. MAPREDUCE-3709. TestDistributedShell is failing. (Hitesh Shah via
  710. mahadev)
  711. MAPREDUCE-3436. JobHistory webapp address should use the host configured
  712. in the jobhistory address. (Ahmed Radwan via sseth)
  713. MAPREDUCE-3815. Fixed MR AM to always use hostnames and never IPs when
  714. requesting containers so that scheduler can give off data local containers
  715. correctly. (Siddarth Seth via vinodkv)
  716. MAPREDUCE-3833. Fixed a bug in reinitiaziling of queues. (Jason Lowe via
  717. acmurthy)
  718. MAPREDUCE-3826. Fixed a bug in RM web-ui which broke sorting. (Jonathan
  719. Eagles via acmurthy)
  720. MAPREDUCE-3823. Ensure counters are calculated only once after a job
  721. finishes. (Vinod Kumar Vavilapalli via sseth)
  722. MAPREDUCE-3827. Changed Counters to use ConcurrentSkipListMap for
  723. performance. (vinodkv via acmurthy)
  724. MAPREDUCE-3822. Changed FS counter computation to use all occurences of
  725. the same FS scheme, instead of randomly using one. (Mahadev Konar via
  726. sseth)
  727. MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times
  728. into the container request table when multiple hosts for a split happen to
  729. be on the same rack. (Siddarth Seth via vinodkv)
  730. MAPREDUCE-3828. Ensure that urls in single-node mode are correct. (sseth
  731. via acmurthy)
  732. MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)
  733. MAPREDUCE-3843. Job summary log file found missing on the RM host
  734. (Anupam Seth via tgraves)
  735. MAPREDUCE-3846. Addressed MR AM hanging issues during AM restart and then
  736. the recovery. (vinodkv)
  737. MAPREDUCE-3802. Added test to validate that AM can crash multiple times and
  738. still can recover successfully after MAPREDUCE-3846. (vinodkv)
  739. MAPREDUCE-3858. Task attempt failure during commit results in task never completing.
  740. (Tom White via mahadev)
  741. MAPREDUCE-3856. Instances of RunningJob class givs incorrect job tracking
  742. urls when mutiple jobs are submitted from same client jvm. (Eric Payne via
  743. sseth)
  744. MAPREDUCE-3880. Changed LCE binary to be 32-bit. (acmurthy)
  745. Release 0.23.0 - 2011-11-01
  746. INCOMPATIBLE CHANGES
  747. MAPREDUCE-2455. Remove deprecated JobTracker.State in favour of
  748. JobTrackerStatus. (tomwhite)
  749. MAPREDUCE-2430. Remove mrunit contrib. (nigel via eli)
  750. MAPREDUCE-2606. Remove IsolationRunner. (Alejandro Abdelnur via eli)
  751. NEW FEATURES
  752. MAPREDUCE-2682. Add "mapred classpath" command to print classpath
  753. for MR applications. (vinodkv via acmurthy)
  754. MAPREDUCE-2107. [Gridmix] Total heap usage emulation in Gridmix.
  755. (Amar Kamat and Ravi Gummadi via amarrk)
  756. MAPREDUCE-2106. [Gridmix] Cumulative CPU usage emulation in Gridmix.
  757. (amarrk)
  758. MAPREDUCE-2543. [Gridmix] High-Ram feature emulation in Gridmix. (amarrk)
  759. MAPREDUCE-2408. [Gridmix] Compression emulation in Gridmix. (amarrk)
  760. MAPREDUCE-2473. Add "mapred groups" command to query the server-side groups
  761. resolved for a user. (Aaron T. Myers via todd)
  762. MAPREDUCE-461. Enable ServicePlugins for the JobTracker.
  763. (Fredrik Hedberg via tomwhite)
  764. MAPREDUCE-2521. Create RPM and Debian packages for MapReduce. Changes
  765. deployment layout to be consistent across the binary tgz, rpm, and deb.
  766. (Eric Yang via omalley)
  767. MAPREDUCE-2323. Add metrics to the fair scheduler. (todd)
  768. MAPREDUCE-2037. Capture intermediate progress, CPU and memory usage for
  769. tasks. (Dick King via acmurthy)
  770. MAPREDUCE-279. MapReduce 2.0. Merging MR-279 branch into trunk. Contributed by
  771. Arun C Murthy, Christopher Douglas, Devaraj Das, Greg Roelofs, Jeffrey
  772. Naisbitt, Josh Wills, Jonathan Eagles, Krishna Ramachandran, Luke Lu, Mahadev
  773. Konar, Robert Evans, Sharad Agarwal, Siddharth Seth, Thomas Graves, and Vinod
  774. Kumar Vavilapalli.
  775. MAPREDUCE-2930. Added the ability to be able to generate graphs from the
  776. state-machine definitions. (Binglin Chang via vinodkv)
  777. MAPREDUCE-2719. Add a simple, DistributedShell, application to illustrate
  778. alternate frameworks on YARN. (Hitesh Shah via acmurthy)
  779. MAPREDUCE-3104. Implemented Application-acls. (vinodkv)
  780. MAPREDUCE-2708. Designed and implemented MR Application Master recovery to
  781. make MR AMs resume their progress after restart. (Sharad Agarwal via vinodkv)
  782. MAPREDUCE-2858. Added a WebApp Proxy for applications. (Robert Evans via
  783. acmurthy)
  784. IMPROVEMENTS
  785. MAPREDUCE-2187. Reporter sends progress during sort/merge. (Anupam Seth via
  786. acmurthy)
  787. MAPREDUCE-2365. Add counters to track bytes (read,written) via
  788. File(Input,Output)Format. (Siddharth Seth via acmurthy)
  789. MAPREDUCE-2680. Display queue name in job client CLI. (acmurthy)
  790. MAPREDUCE-2679. Minor changes to sync trunk with MR-279 branch. (acmurthy)
  791. MAPREDUCE-2400. Remove Cluster's dependency on JobTracker via a
  792. ServiceProvider for the actual implementation. (tomwhite via acmurthy)
  793. MAPREDUCE-2596. [Gridmix] Summarize Gridmix runs. (amarrk)
  794. MAPREDUCE-2563. [Gridmix] Add High-Ram emulation system tests to
  795. Gridmix. (Vinay Kumar Thota via amarrk)
  796. MAPREDUCE-2104. [Rumen] Add Cpu, Memory and Heap usages to
  797. TraceBuilder's output. (amarrk)
  798. MAPREDUCE-2554. [Gridmix] Add distributed cache emulation system tests
  799. to Gridmix. (Vinay Kumar Thota via amarrk)
  800. MAPREDUCE-2543. [Gridmix] High-Ram feature emulation testcase. (amarrk)
  801. MAPREDUCE-2469. Task counters should also report the total heap usage of
  802. the task. (Ravi Gummadi and Amar Ramesh Kamat via amarrk)
  803. MAPREDUCE-2544. [Gridmix] Add compression emulation system tests to
  804. Gridmix. (Vinay Kumar Thota via amarrk)
  805. MAPREDUCE-2517. [Gridmix] Add system tests to Gridmix.
  806. (Vinay Kumar Thota via amarrk)
  807. MAPREDUCE-2492. The new MapReduce API should make available task's
  808. progress to the task. (amarrk)
  809. MAPREDUCE-2153. Bring in more job configuration properties in to the trace
  810. file. (Rajesh Balamohan via amarrk)
  811. MAPREDUCE-1461. Feature to instruct rumen-folder utility to skip jobs worth
  812. of specific duration. (Rajesh Balamohan via amarrk)
  813. MAPREDUCE-2172. Added test-patch.properties required by test-patch.sh
  814. (nigel)
  815. MAPREDUCE-2156. Raid-aware FSCK. (Patrick Kling via dhruba)
  816. MAPREDUCE-2215. A more elegant FileSystem#listCorruptFileBlocks API
  817. (RAID changes) (Patrick Kling via hairong)
  818. MAPREDUCE-1831. BlockPlacement policy for HDFS-RAID.
  819. (Scott Chen via dhruba)
  820. MAPREDUCE-1906. Lower minimum heartbeat interval for TaskTracker
  821. (Scott Carey and Todd Lipcon via todd)
  822. MAPREDUCE-1382. MRAsyncDiscService should tolerate missing local.dir.
  823. (Zheng Shao and tomwhite via tomwhite)
  824. MAPREDUCE-2263. MapReduce side of HADOOP-6904: RPC compatibility.
  825. (hairong)
  826. MAPREDUCE-1706. Log RAID recoveries on HDFS. (schen)
  827. MAPREDUCE-2334. Update BlockPlacementPolicyRaid for the new method
  828. in BlockPlacementPolicy. (szetszwo)
  829. MAPREDUCE-2254. Allow setting of end-of-record delimiter for
  830. TextInputFormat (Ahmed Radwan via todd)
  831. MAPREDUCE-1927. Unit test for HADOOP-6835 (concatenated gzip support).
  832. (Greg Roelofs via tomwhite)
  833. MAPREDUCE-2206. The task-cleanup tasks should be optional. (schen)
  834. MAPREDUCE-2225. MultipleOutputs should not require the use of 'Writable'.
  835. (Harsh J Chouraria via tomwhite)
  836. MAPREDUCE-1811. Job.monitorAndPrintJob() should print status of the job
  837. at completion. (Harsh J Chouraria via tomwhite)
  838. MAPREDUCE-993. bin/hadoop job -events <jobid> <from-event-#> <#-of-events>
  839. help message is confusing. (Harsh J Chouraria via tomwhite)
  840. MAPREDUCE-2302. Add static factory methods in GaloisField. (schen)
  841. MAPREDUCE-2351. mapred.job.tracker.history.completed.location should
  842. support an arbitrary filesystem URI. (tomwhite)
  843. MAPREDUCE-2239. BlockPlacementPolicyRaid should call getBlockLocations
  844. only when necessary. (schen)
  845. MAPREDUCE-2331. Add coverage of task graph servlet to fair scheduler system
  846. test. (todd)
  847. MAPREDUCE-2367. Allow using a file to exclude certain tests from build.
  848. (todd)
  849. MAPREDUCE-2202. Generalize CLITest structure and interfaces to faciliate
  850. upstream adoption (e.g. for web or system testing). (cos)
  851. MAPREDUCE-2420. JobTracker should be able to renew delegation token over
  852. HTTP (Boris Shkolnik via jitendra)
  853. MAPREDUCE-2474. Add docs to the new API Partitioner on how to access the
  854. Job Configuration. (Harsh J Chouraria via todd)
  855. MAPREDUCE-2475. Disable IPV6 for junit tests. (suresh srinivas via mahadev)
  856. MAPREDUCE-2422. Removed unused internal methods from DistributedCache.
  857. (tomwhite)
  858. MAPREDUCE-2456. Log the reduce taskID and associated TaskTrackers with
  859. failed fetch notifications in the JobTracker log.
  860. (Jeffrey Naisbitt via cdouglas)
  861. MAPREDUCE-869. Documentation for config to set map/reduce task environment
  862. (Alejandro Abdelnur via todd)
  863. MAPREDUCE-2410. Add entry to streaming FAQ about how streaming reducers
  864. receive keys. (Harsh J Chouraria via todd)
  865. MAPREDUCE-2499. MR part of HADOOP-7291. (eli)
  866. MAPREDUCE-2497. Missing spaces in error messages. (eli)
  867. MAPREDUCE-2502. JobSubmitter should use mapreduce.job.maps instead of
  868. its deprecated equivalent. (eli via todd)
  869. MAPREDUCE-2381. JobTracker instrumentation not consistent about error
  870. handling. (Philip Zeyliger via tomwhite)
  871. MAPREDUCE-2449. Allow for command line arguments when performing
  872. "Run on Hadoop" action in Eclipse plugin. (Jeff Zemerick via todd)
  873. MAPREDUCE-2483. Remove duplication of jars between Hadoop subprojects
  874. from build artifacts. (Eric Yang via omalley)
  875. MAPREDUCE-2372. TaskLogAppender mechanism shouldn't be set up in
  876. log4j.properties (todd)
  877. MAPREDUCE-2516. Rename webinterface.private.actions to
  878. mapreduce.jobtracker.webinterface.trusted (Ari Rabkin via todd)
  879. MAPREDUCE-2459. Cache HAR filesystem metadata. (Mac Yang via mahadev)
  880. HADOOP-7259. Contrib modules should include the build.properties from
  881. the enclosing hadoop directory. (omalley)
  882. MAPREDUCE-2494. Order distributed cache deletions by LRU. (Robert Joseph
  883. Evans via cdouglas)
  884. MAPREDUCE-2452. Makes the cancellation of delegation tokens happen in a
  885. separate thread. (ddas)
  886. HADOOP-7106. Reorganize project SVN layout to "unsplit" the projects.
  887. (todd, nigel)
  888. MAPREDUCE-2249. Check the reflexive property of Counters objects when
  889. comparing equality. (Devaraj K via todd)
  890. MAPREDUCE-2623. Update ClusterMapReduceTestCase to use
  891. MiniDFSCluster.Builder (Harsh J Chouraria via eli)
  892. MAPREDUCE-2602. Allow setting of end-of-record delimiter for
  893. TextInputFormat for the old API. (Ahmed Radwan via todd)
  894. MAPREDUCE-2705. Permits parallel multiple task launches.
  895. (Thomas Graves via ddas)
  896. MAPREDUCE-2489. Jobsplits with random hostnames can make the queue
  897. unusable (jeffrey naisbit via mahadev)
  898. MAPREDUCE-2854. update INSTALL with config necessary run mapred on yarn.
  899. (thomas graves via mahadev)
  900. MAPREDUCE-2701. app/Job.java needs UGI for the user that launched it.
  901. (Robert Evans via mahadev)
  902. MAPREDUCE-2652. Enabled multiple NMs to be runnable on a single node by
  903. making shuffle service port to be truely configurable. (Robert Evans via
  904. vinodkv)
  905. MAPREDUCE-2735. Add an applications summary log to ResourceManager.
  906. (Thomas Graves via acmurthy)
  907. MAPREDUCE-2697. Enhance CapacityScheduler to cap concurrently running
  908. applications per-queue & per-user. (acmurthy)
  909. Configuration changes:
  910. add yarn.capacity-scheduler.maximum-am-resource-percent
  911. MAPREDUCE-2774. Add startup message to ResourceManager & NodeManager on
  912. startup. (Venu Gopala Rao via acmurthy)
  913. MAPREDUCE-2655. Add audit logs to ResourceManager and NodeManager. (Thomas
  914. Graves via acmurthy)
  915. MAPREDUCE-2864. Normalize configuration variable names for YARN. (Robert
  916. Evans via acmurthy)
  917. MAPREDUCE-2690. Web-page for FifoScheduler. (Eric Payne via acmurthy)
  918. MAPREDUCE-2711. Update TestBlockPlacementPolicyRaid for the new namesystem
  919. and block management APIs. (szetszwo)
  920. MAPREDUCE-2933. Change allocate call to return ContainerStatus for
  921. completed containers rather than Container. (acmurthy)
  922. MAPREDUCE-2675. Reformat JobHistory Server main page to be more
  923. useful. (Robert Joseph Evans via vinodkv).
  924. MAPREDUCE-2896. Simplify all apis to in
  925. org.apache.hadoop.yarn.api.records.* to be get/set only. Added javadocs to
  926. all public records. (acmurthy)
  927. MAPREDUCE-2676. MR-279: JobHistory Job page needs reformatted. (Robert Evans via
  928. mahadev)
  929. MAPREDUCE-2899. Replace major parts of ApplicationSubmissionContext with a
  930. ContainerLaunchContext (Arun Murthy via mahadev)
  931. MAPREDUCE-2966. Added ShutDown hooks for MRV2 processes so that they can
  932. gracefully exit. (Abhijit Suresh Shingate via vinodkv)
  933. MAPREDUCE-2672. MR-279: JobHistory Server needs Analysis this job.
  934. (Robert Evans via mahadev)
  935. MAPREDUCE-2965. Streamlined the methods hashCode(), equals(), compareTo()
  936. and toString() for all IDs. (Siddharth Seth via vinodkv)
  937. MAPREDUCE-2726. Added job-file to the AM and JobHistoryServer web
  938. interfaces. (Jeffrey Naisbitt via vinodkv)
  939. MAPREDUCE-2880. Improve classpath-construction for mapreduce AM and
  940. containers. (Arun C Murthy via vinodkv)
  941. MAPREDUCE-3055. Simplified ApplicationAttemptId passing to
  942. ApplicationMaster via environment variable. (vinodkv)
  943. MAPREDUCE-3092. Removed a special comparator for JobIDs in JobHistory as
  944. JobIDs are already comparable. (Devaraj K via vinodkv)
  945. MAPREDUCE-3099. Add docs for setting up a single node MRv2 cluster.
  946. (mahadev)
  947. MAPREDUCE-3001. Added task-specific counters to AppMaster and JobHistory
  948. web-UIs. (Robert Joseph Evans via vinodkv)
  949. MAPREDUCE-3098. Fixed RM and MR AM to report YarnApplicationState and
  950. application's FinalStatus separately. (Hitesh Shah via vinodkv)
  951. MAPREDUCE-2889. Added documentation for writing new YARN applications.
  952. (Hitesh Shah via acmurthy)
  953. MAPREDUCE-3134. Added documentation the CapacityScheduler. (acmurthy)
  954. MAPREDUCE-3013. Removed YarnConfiguration.YARN_SECURITY_INFO and its usage
  955. as it doesn't affect security any more. (vinodkv)
  956. MAPREDUCE-2907. Changed log level for various messages in ResourceManager
  957. from INFO to DEBUG. (Ravi Prakash via vinodkv)
  958. MAPREDUCE-2702. Added a new API in OutputCommitter for recovering
  959. the outputs of tasks from a crashed job so as to support MR Application
  960. Master recovery. (Sharad Agarwal and Arun C Murthy via vinodkv)
  961. MAPREDUCE-2738. Added the missing cluster level statistics on the RM web
  962. UI. (Robert Joseph Evans via vinodkv)
  963. MAPREDUCE-2988. Reenabled TestLinuxContainerExecutor reflecting the
  964. current NodeManager code. (Robert Joseph Evans via vinodkv)
  965. MAPREDUCE-3161. Improved some javadocs and fixed some typos in
  966. YARN. (Todd Lipcon via vinodkv)
  967. MAPREDUCE-3148. Ported MAPREDUCE-2702 to old mapred api for aiding task
  968. recovery. (acmurthy)
  969. MAPREDUCE-3133. Running a set of methods in a Single Test Class.
  970. (Jonathan Eagles via mahadev)
  971. MAPREDUCE-3059. QueueMetrics do not have metrics for aggregate
  972. containers-allocated and aggregate containers-released.
  973. (Devaraj K via mahadev)
  974. MAPREDUCE-3187. Add names for various unnamed threads in MR2.
  975. (Todd Lipcon and Siddharth Seth via mahadev)
  976. MAPREDUCE-3136. Added documentation for setting up Hadoop clusters in both
  977. non-secure and secure mode for both HDFS & YARN. (acmurthy)
  978. MAPREDUCE-3068. Added a whitelist of environment variables for containers
  979. from the NodeManager and set MALLOC_ARENA_MAX for all daemons and
  980. containers. (Chris Riccomini via acmurthy)
  981. MAPREDUCE-3144. Augmented JobHistory with the information needed for
  982. serving aggregated logs. (Siddharth Seth via vinodkv)
  983. MAPREDUCE-3163. JobClient spews errors when killing MR2 job.
  984. (mahadev)
  985. MAPREDUCE-3239. Use new createSocketAddr API in MRv2 to give better
  986. error messages on misconfig (Todd Lipcon via mahadev)
  987. MAPREDUCE-2747. Cleaned up LinuxContainerExecutor binary sources and changed
  988. the configuration to use yarn names. (Robert Joseph Evans via vinodkv)
  989. MAPREDUCE-3205. Fix memory specifications to be physical rather than
  990. virtual, allowing for a ratio between the two to be configurable. (todd
  991. via acmurthy)
  992. MAPREDUCE-2986. Fixed MiniYARNCluster to support multiple NodeManagers.
  993. (Anupam Seth via vinodkv)
  994. MAPREDUCE-2736. Remove unused contrib components dependent on MR1. (eli)
  995. MAPREDUCE-2989. Modified JobHistory to link to task and AM logs from the
  996. JobHistoryServer. (Siddharth Seth via vinodkv)
  997. MAPREDUCE-3014. Rename and invert logic of '-cbuild' profile to 'native' and off
  998. by default. (tucu)
  999. MAPREDUCE-3171. normalize nodemanager native code compilation with common/hdfs
  1000. native. (tucu)
  1001. MAPREDUCE-3146. Added a MR specific command line to dump logs for a
  1002. given TaskAttemptID. (Siddharth Seth via vinodkv)
  1003. MAPREDUCE-3275. Added documentation for AM WebApp Proxy. (Robert Evans via
  1004. acmurthy)
  1005. MAPREDUCE-3322. Added a better index.html and an brief overview of YARN
  1006. architecture. (acmurthy)
  1007. OPTIMIZATIONS
  1008. MAPREDUCE-2026. Make JobTracker.getJobCounters() and
  1009. JobInProgress.getCounters() aquire locks in a shorter time period.
  1010. (Joydeep Sen Sarma via schen)
  1011. MAPREDUCE-2740. MultipleOutputs in new API creates needless
  1012. TaskAttemptContexts. (todd)
  1013. MAPREDUCE-901. Efficient framework counters. (llu via acmurthy)
  1014. MAPREDUCE-2629. Workaround a JVM class loading quirk which prevents
  1015. JIT compilation of inner classes methods in ReduceContextImpl.
  1016. BUG FIXES
  1017. MAPREDUCE-2603. Disable High-Ram emulation in system tests.
  1018. (Vinay Kumar Thota via amarrk)
  1019. MAPREDUCE-2539. Fixed NPE in getMapTaskReports in JobClient. (Robert Evans via
  1020. acmurthy)
  1021. MAPREDUCE-1978. Rumen TraceBuilder should provide recursive
  1022. input folder scanning.
  1023. MAPREDUCE-2416. Remove the restriction of specifying group names in
  1024. users-list file for Gridmix in RoundRobinUserResolver mode.
  1025. MAPREDUCE-2417. Fix Gridmix in RoundRobinUserResolver mode to
  1026. map testing/proxy users to unique users in a trace.
  1027. MAPREDUCE-2307. Exception thrown in Jobtracker logs, when the Scheduler
  1028. configured is FairScheduler. (Devaraj K via matei)
  1029. MAPREDUCE-2199. build is broken 0.22 branch creation. (cos)
  1030. MAPREDUCE-1752. Implement getFileBlockLocations in HarFilesystem.
  1031. (Patrick Kling via dhruba)
  1032. MAPREDUCE-2155. RaidNode should optionally use the mapreduce jobs to
  1033. fix missing blocks. (Patrick Kling via dhruba)
  1034. MAPREDUCE-1334. Fix TestIndexUpdater by ignoring _SUCCESS file in HDFS.
  1035. (Kay Kay via yhemanth)
  1036. MAPREDUCE-2232. Add missing methods to TestMapredGroupMappingServiceRefresh.
  1037. (Todd Lipcon via eli)
  1038. MAPREDUCE-2271. Fix TestSetupTaskScheduling failure on trunk.
  1039. (Liyin Liang via todd)
  1040. MAPREDUCE-2290. Fix compilation error in TestTaskCommit. (eli)
  1041. MAPREDUCE-2294. Fix compilation error in mumak. (todd)
  1042. MAPREDUCE-2300. Fix TestUmbilicalProtocolWithJobToken on trunk after
  1043. HADOOP-6904. (todd)
  1044. MAPREDUCE-2296. Fix references to misspelled method name
  1045. getProtocolSigature (todd)
  1046. MAPREDUCE-2311. Fix TestFairScheduler failure (schen)
  1047. MAPREDUCE-1996. API: Reducer.reduce() method detail misstatement.
  1048. (Harsh J Chouraria via tomwhite)
  1049. MAPREDUCE-2203. Wrong javadoc for TaskRunner's appendJobJarClasspaths
  1050. method. (Jingguo Yao via tomwhite)
  1051. MAPREDUCE-2074. Task should fail when symlink creation fails.
  1052. (Priyo Mustafi via tomwhite)
  1053. MAPREDUCE-1242. Chain APIs error misleading.
  1054. (Harsh J Chouraria via tomwhite)
  1055. MAPREDUCE-2379. Adds missing DistributedCache configurations in
  1056. mapred-default.xml (Todd Lipcon via amareshwari)
  1057. MAPREDUCE-2348. Disable mumak tests on trunk since they currently time out
  1058. (todd)
  1059. MAPREDUCE-2395. TestBlockFixer timing out on trunk. (Ramkumar Vadali via
  1060. todd)
  1061. MAPREDUCE-2426. Make TestFairSchedulerSystem fail with more verbose output
  1062. (todd)
  1063. MAPREDUCE-2448. NoSuchMethodError:
  1064. org.apache.hadoop.hdfs.TestDatanodeBlockScanner.corruptReplica(..) (eli)
  1065. MAPREDUCE-2460. Fix flaky test TestFairSchedulerSystem. (todd)
  1066. MAPREDUCE-2451. Log the details from health check script at the
  1067. JobTracker. (Thomas Graves via cdouglas)
  1068. MAPREDUCE-2467. HDFS-1052 changes break the raid contrib module in
  1069. MapReduce. (suresh srinivas via mahadev)
  1070. MAPREDUCE-2258. IFile reader closes stream and compressor in wrong order.
  1071. (todd via tomwhite)
  1072. MAPREDUCE-2518. The t flag is missing in distcp help message. (Wei Yongjun
  1073. via szetszwo)
  1074. MAPREDUCE-2514. Fix typo in TaskTracker ReinitTrackerAction log message.
  1075. (Jonathan Eagles via cdouglas)
  1076. MAPREDUCE-2490. Add logging to graylist and blacklist activity to aid
  1077. diagnosis of related issues. (Jonathan Eagles via cdouglas)
  1078. MAPREDUCE-2495. exit() the TaskTracker when the distributed cache cleanup
  1079. thread dies. (Robert Joseph Evans via cdouglas)
  1080. MAPREDUCE-2470. Fix NPE in RunningJobs::getCounters. (Robert Joseph Evans
  1081. via cdouglas)
  1082. MAPREDUCE-2536. Update FsShell -mv command usage in TestMRCLI. (Daryn
  1083. Sharp via szetszwo)
  1084. MAPREDUCE-2529. Add support for regex-based shuffle metric counting
  1085. exceptions. (Thomas Graves via cdouglas)
  1086. MAPREDUCE-2559. ant binary fails due to missing c++ lib dir. (eli)
  1087. MAPREDUCE-2573. Fix new findbugs warning introduced by MAPREDUCE-2494.
  1088. (Robert Joseph Evans via todd)
  1089. MAPREDUCE-2581. Spelling errors in log messages. (Tim Sell via eli)
  1090. MAPREDUCE-2588. Change raid to the new DataTransferProtocol API. (szetszwo)
  1091. MAPREDUCE-2576. Typo in comment in SimulatorLaunchTaskAction.java.
  1092. (Tim Sell via jghoman)
  1093. MAPREDUCE-2550. Fix bin/mapred to work properly from within a source
  1094. checkout (Eric Yang via todd)
  1095. MAPREDUCE-2620. Update RAID for HDFS-2087. (szetszwo)
  1096. MAPREDUCE-2624. Update RAID for HDFS-2107. (szetszwo)
  1097. MAPREDUCE-2670. Fixing spelling mistake in FairSchedulerServlet.java. (eli)
  1098. MAPREDUCE-2710. Update JobSubmitter.printTokens(..) for HDFS-2161.
  1099. (szetszwo)
  1100. MAPREDUCE-2409. DistributedCache maps files and archives to the same path,
  1101. despite semantic incompatibility. (Siddharth Seth via cdouglas)
  1102. MAPREDUCE-2575. TestMiniMRDFSCaching fails if test.build.dir is set
  1103. to something other than build/test (Thomas Graves via mahadev)
  1104. MAPREDUCE-2622. Remove the last remaining reference to the deprecated
  1105. configuration "io.sort.mb". (Harsh J Chouraria via todd)
  1106. MAPREDUCE-2732. Remove directly accessing FSNamesystem.LOG from
  1107. TestCopyFiles and TestDistCh. (szetszwo)
  1108. MAPREDUCE-2463. Job history files are not moved to done folder when job
  1109. history location is hdfs. (Devaraj K via szetszwo)
  1110. MAPREDUCE-2243. Close streams propely in a finally-block to avoid leakage
  1111. in CompletedJobStatusStore, TaskLog, EventWriter and TotalOrderPartitioner.
  1112. (Devaraj K via szetszwo)
  1113. MAPREDUCE-2741. Make ant build system work with hadoop-common JAR
  1114. generated by Maven. (Alejandro Abdelnur via tomwhite)
  1115. MAPREDUCE-2760. mapreduce.jobtracker.split.metainfo.maxsize typoed
  1116. in mapred-default.xml. (todd via eli)
  1117. MAPREDUCE-2797. Update mapreduce tests and RAID for HDFS-2239. (szetszwo)
  1118. MAPREDUCE-2805. Update RAID for HDFS-2241. (szetszwo)
  1119. MAPREDUCE-2837. Ported bug fixes from y-merge to prepare for MAPREDUCE-279
  1120. merge. (acmurthy)
  1121. MAPREDUCE-2541. Fixed a race condition in IndexCache.removeMap. (Binglin
  1122. Chang via acmurthy)
  1123. MAPREDUCE-2458. Rename sanitized pom.xml in build directory to work around IDE
  1124. bug (Luke Lu via mahadev)
  1125. MAPREDUCE-279. Fix in MR-279 branch. Clear application notification if sent once
  1126. to NodeManager (mahadev)
  1127. MAPREDUCE-2433. YARNApplicationConstants hard code app master jar version (Luke
  1128. Lu via mahadev)
  1129. MAPREDUCE-279. Fix in MR-279 branch. Implement restart for resource manager
  1130. phase 1 - Helper classes to store and restore the data structures. (mahadev)
  1131. MAPREDUCE-2414. Change MRv2 to use generic interfaces. (Siddharth Seth via
  1132. acmurthy)
  1133. MAPREDUCE-279. Fix in MR-279 branch. Implement health-checks for the node -
  1134. server side(ResourceManager) changes. (vinodkv)
  1135. MAPREDUCE-2405: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2)
  1136. (Greg Roelofs via mahadev)
  1137. MAPREDUCE-279. Fix in MR-279 branch. Implementing Containers' memory monitoring.
  1138. (vinodkv)
  1139. MAPREDUCE-2440. Name clashes in TypeConverter (luke via mahadev)
  1140. MAPREDUCE-279. Fix in MR-279 branch. Add fail count to the command line of the
  1141. application master. (mahadev)
  1142. MAPREDUCE-2424. Polish uber-AppMaster: add uber-AM counters and GUI indicators.
  1143. (Greg Roelofs via mahadev)
  1144. MAPREDUCE-2405. Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2).
  1145. (Greg Roelofs and Sharad Agarwal via mahadev)
  1146. MAPREDUCE-279. Fix in MR-279 branch. Fix YarnRemoteException to give more
  1147. details. (Siddharth Seth via mahadev)
  1148. MAPREDUCE-279. Fix in MR-279 branch. WebApp for Job History (Krishna
  1149. Ramachandran via mahadev)
  1150. MAPREDUCE-279. Fix in MR-279 branch. Install sanitized poms for downstream
  1151. sanity (Luke Lu via mahadev)
  1152. MAPREDUCE-279. Fix in MR-279 branch. Add HistoryCleanerService to Job History
  1153. server. (Krishna Ramachandran via sharad)
  1154. MAPREDUCE-279. Fix in MR-279 branch. Implement 'bin/mapred job -list' and
  1155. 'bin/mapred job -list-active-trackers'. (acmurthy)
  1156. MAPREDUCE-279. Fix in MR-279 branch. Implement 'bin/mapred queue [-info
  1157. [-showJobs]] [-list] and enhanced 'bin/mapred job -list' to show queue and
  1158. ApplicationMaster information. (acmurthy)
  1159. MAPREDUCE-279. Fix in MR-279 branch. Fixed computation of user-limits at
  1160. runtime. (acmurthy)
  1161. MAPREDUCE-279. Fix in MR-279 branch. Added functionality to refresh queues at
  1162. runtime via the 'bin/yarn rmadmin' command. (acmurthy)
  1163. MAPREDUCE-279. Fix in MR-279 branch. Added functionality to stop/start queues.
  1164. (acmurthy)
  1165. MAPREDUCE-279. Fix in MR-279 branch. Introducing web-UI for NodeManager and
  1166. linking it from RM UI. (vinodkv)
  1167. MAPREDUCE-279. Fix in MR-279 branch. Fix race condition in TestJobHistoryEvents
  1168. and TestJobHistoryParsing. (sharad)
  1169. MAPREDUCE-279. Fix in MR-279 branch. Add Containers' logs' view to NM UI and
  1170. link it from AM UI. (vinodkv)
  1171. MAPREDUCE-279. Fix in MR-279 branch. Add ACLs for queues and command-line
  1172. utilities for viewing them. (acmurthy)
  1173. MAPREDUCE-279. Fix in MR-279 branch. Recovery of MR Application Master from
  1174. failures. (sharad)
  1175. MAPREDUCE-279. Fix in MR-279 branch. Added support High-RAM applications in
  1176. CapacityScheduler. (acmurthy)
  1177. MAPREDUCE-279. Fix in MR-279 branch. Completing the ZooKeeper Store for
  1178. ResourceManager state. (mahadev)
  1179. MAPREDUCE-279. Fix in MR-279 branch. Reorient container localization to be
  1180. per-container rather than per-application. (cdouglas)
  1181. MAPREDUCE-279. Fix in MR-279 branch. Fix file creation in
  1182. JobHistoryEventHandler. (sharad)
  1183. MAPREDUCE-279. Fix in MR-279 branch. Disable ContainerMonitoring for non-linux
  1184. systems. (vinodkv)
  1185. MAPREDUCE-279. Fix in MR-279 branch. Fix container launch w/ inconsistent
  1186. credential file naming. (cdouglas)
  1187. MAPREDUCE-2434. Metrics for ResourceManager. (Luke Lu via acmurthy)
  1188. MAPREDUCE-279. Fix in MR-279 branch. RM Restart Phase 2 - Completed the recovery
  1189. of components in the RM (mahadev)
  1190. MAPREDUCE-279. Fix in MR-279 branch. Fix to send finish application event only
  1191. when the application is finished (mahadev)
  1192. MAPREDUCE-2462. Write job conf along with JobHistory, other minor improvements.
  1193. (Siddharth Seth via sharad)
  1194. MAPREDUCE-279. Fix in MR-279 branch. Implement 'delay scheduling' for better
  1195. locality in CapacityScheduler and improved high-ram applications. (acmurthy)
  1196. MAPREDUCE-279. Fix in MR-279 branch. Implement Job Acls in MR Application
  1197. Master. (sharad)
  1198. MAPREDUCE-279. Fix in MR-279 branch. Moving userlogs out of container work-dir
  1199. into a separate directory structure. (vinodkv)
  1200. MAPREDUCE-279. Fix in MR-279 branch. Completing RM Restart. Completed Phase 3 of
  1201. making sure events are logged and restored (mahadev)
  1202. MAPREDUCE-2468. Add metrics for NM Shuffle. (Luke Lu via cdouglas)
  1203. MAPREDUCE-279. Fix in MR-279 branch. Adding user log handling for YARN. Making
  1204. NM put the user-logs on DFS and providing log-dump tools. (vinodkv)
  1205. MAPREDUCE-279. Fix in MR-279 branch. Fixing three tight-loops in RM that are
  1206. causing high cpu-usage. (vinodkv)
  1207. MAPREDUCE-279. Fix in MR-279 branch. Replacing FileContext usage with FileSystem
  1208. to work around security authentication issues with FileContext against a secure
  1209. DFS. (vinodkv)
  1210. MAPREDUCE-279. Fix in MR-279 branch. Client reconnect to restarted AM. (sharad)
  1211. MAPREDUCE-279. Fix in MR-279 branch. Fix refreshProxy in ClientServiceDelegate.
  1212. (sharad)
  1213. MAPREDUCE-279. Fix in MR-279 branch. Fix Null Pointer in TestUberAM. (sharad)
  1214. MAPREDUCE-2478. Improve history server. (Siddharth Seth via sharad)
  1215. MAPREDUCE-279. Fix in MR-279 branch. Improved TestJobHistoryEvents and
  1216. TestJobHistoryParsing. (sharad)
  1217. MAPREDUCE-279. Fix in MR-279 branch. Fix NM to use multiple disks for local
  1218. files and the userlogs. (vinodkv)
  1219. MAPREDUCE-2480: MR App should not depend on hard coded version of shuffle (luke
  1220. lu via mahadev)
  1221. MAPREDUCE-279. Fix in MR-279 branch. Propagate error back to client in case of a
  1222. job submission failure (mahadev)
  1223. MAPREDUCE-279. Fix in MR-279 branch. Fix assembly to add mapreduce shell scripts
  1224. to the assembly package. (mahadev)
  1225. MAPREDUCE-279. Fix in MR-279 branch. Fix TestQueueMetrics. (Luke Lu via sharad)
  1226. MAPREDUCE-279. Fix in MR-279 branch. Removal of stale application-log dirs from
  1227. NM local disks. (vinodkv)
  1228. MAPREDUCE-279. Fix in MR-279 branch. Add license header and minor cleanup in
  1229. history server. (Siddharth Seth via sharad)
  1230. MAPREDUCE-279. Fix in MR-279 branch. Minor fix for install instructions.
  1231. (mahadev)
  1232. MAPREDUCE-279. Fix in MR-279 branch. Fix a race in MR task that was causing MR
  1233. containers to overwrite each other's job.xml. Also fix leaking attempt-dirs in
  1234. app-local-dir. (vinodkv)
  1235. MAPREDUCE-279. Fix in MR-279 branch. Adding valid state to ASM on a finish when
  1236. its already completed and also disble UberAM. (mahadev)
  1237. MAPREDUCE-279. Fix in MR-279 branch. Fixed CS user limits. (acmurthy)
  1238. MAPREDUCE-279. Fix in MR-279 branch. Fixed reservation's bad interaction with
  1239. delay scheduling in CS. (acmurthy)
  1240. MAPREDUCE-279. Fix in MR-279 branch. Support mapreduce old (0.20) APIs. (sharad)
  1241. MAPREDUCE-279. Fix in MR-279 branch. Support fail-fast for MR jobs. (ddas)
  1242. MAPREDUCE-279. Fix in MR-279 branch. Fix for clearing container requests on an
  1243. AM failure and add tostring methods to taskids and taskattemptids for better
  1244. grep support. (mahadev)
  1245. MAPREDUCE-279. Fix in MR-279 branch. Speed up communication between MR AM and RM
  1246. by relying on a new config rather than AM_EXPIRY_INTERVAL which is too large.
  1247. (acmurthy)
  1248. MAPREDUCE-279. Fix in MR-279 branch. Fix calculation of maximum capacity to use
  1249. parent's absolute-capacity rather than the leaf queue's absolute-capacity.
  1250. (acmurthy)
  1251. MAPREDUCE-279. Fix in MR-279 branch. Fixing a bug in JobIDPbImpl that's causing
  1252. AM to crash randomly. (vinodkv)
  1253. MAPREDUCE-279. Fix in MR-279 branch. Fix calculation of maximum capacity in
  1254. ParentQueue to use its parent's absolute-capacity rather than its own
  1255. absolute-capacity. (acmurthy)
  1256. MAPREDUCE-279. Fix in MR-279 branch. Launching bin/yarn and bin/mapred only
  1257. *once* in AM for constructing classpaths to avoid multiple forks and huge vmem
  1258. usage by AM. (vinodkv)
  1259. MAPREDUCE-279. Fix in MR-279 branch. Fix CapacityScheduler to release unused
  1260. reservations on application completion. (acmurthy)
  1261. MAPREDUCE-279. Fix in MR-279 branch. Fix CapacityScheduler (LeafQueue) to not
  1262. allocate DATA_LOCAL containers when they are not required on the rack.
  1263. (acmurthy)
  1264. MAPREDUCE-279. Fix in MR-279 branch. Makes uber-task disabled by default (ddas)
  1265. MAPREDUCE-279. Fix in MR-279 branch. Make logging and memory for AM configurable
  1266. for the user via command line (mahadev)
  1267. MAPREDUCE-279. Fix in MR-279 branch. Fixing a bug in previous patch (r1103657).
  1268. Now bin/yarn truly shouldn't be launched multiple times in a single AM.
  1269. (vinodkv)
  1270. MAPREDUCE-279. Fix in MR-279 branch. Fixing a bug to do with setting the staging
  1271. dir. (ddas)
  1272. MAPREDUCE-279. Fix in MR-279 branch. Fixed Composite Service to shutdown
  1273. services if an error occurs on starting any one of those (mahadev & chris)
  1274. MAPREDUCE-279. Fix in MR-279 branch. Fix the tests to use jvm fork mode to avoid
  1275. errors in shutting down services (sidharth seth)
  1276. MAPREDUCE-2500. PB factories are not thread safe (Siddharth Seth via mahadev)
  1277. MAPREDUCE-2504. race in JobHistoryEventHandler stop (Siddharth Seth via mahadev)
  1278. MAPREDUCE-279. Fix in MR-279 branch. Fix job hang if the AM launch fails.
  1279. (mahadev)
  1280. MAPREDUCE-2509. Fix NPE in UI for pending attempts. (Luke Lu via mahadev)
  1281. MAPREDUCE-279. Fix in MR-279 branch. Add junit jar to lib in assembly (mahadev
  1282. and luke)
  1283. MAPREDUCE-279. Fix in MR-279 branch. Distributed cache bug fix to pass Terasort.
  1284. (vinodkv)
  1285. MAPREDUCE-279. Fix in MR-279 branch. Fix null pointer exception in kill task
  1286. attempt (mahadev)
  1287. MAPREDUCE-279. Fix in MR-279 branch. Refactored RMContainerAllocator to release
  1288. unused containers. (sharad)
  1289. MAPREDUCE-279. Fix in MR-279 branch. Changed Scheduler to return available limit
  1290. to AM in the allocate api. (acmurthy)
  1291. MAPREDUCE-279. Fix in MR-279 branch. Fix nodemanager expiry to not throw OOM.
  1292. (mahadev)
  1293. MAPREDUCE-279. Fix in MR-279 branch. Use DefaultContainerExecutor for
  1294. integration tests. (cdouglas)
  1295. MAPREDUCE-279. Fix in MR-279 branch. Fix NPE in test case (mahadev)
  1296. MAPREDUCE-279. Fix in MR-279 branch. Fix for regression on the scheduling of
  1297. reduces before maps are done (ddas)
  1298. MAPREDUCE-279. Fix in MR-279 branch. Fix distributed-cache related bugs.
  1299. (vinodkv)
  1300. MAPREDUCE-279. Fix in MR-279 branch. Ensure unused containers released by AM are
  1301. correctly counted for queue-capacity. (acmurthy)
  1302. MAPREDUCE-279. Fix in MR-279 branch. Fix TestRuntimeEstimators (Siddharth Seth
  1303. via ddas)
  1304. MAPREDUCE-279. Fix in MR-279 branch. Fix queue refresh to correctly record newly
  1305. added queues in CapacityScheduler. (acmurthy)
  1306. MAPREDUCE-279. Fix in MR-279 branch. Added metrics for tracking reservations in
  1307. CapacityScheduler. (Luke Lu via acmurthy)
  1308. MAPREDUCE-2522. Security for JobHistory service. (Siddharth Seth via mahadev)
  1309. MAPREDUCE-2534. Fix CI breaking hard coded version in jobclient pom. (Luke Lu
  1310. via mahadev)
  1311. MAPREDUCE-279. Fix in MR-279 branch. Add public cache. (cdouglas)
  1312. MAPREDUCE-279. Fix in MR-279 branch. Made number of RPC server threads
  1313. configurable. (acmurthy)
  1314. MAPREDUCE-279. Fix in MR-279 branch. Added acl check for RMAdmin. (acmurthy)
  1315. MAPREDUCE-279. Fix in MR-279 branch. Adding job kill for any state that the job
  1316. is in with access control. (mahadev)
  1317. MAPREDUCE-279. Fix in MR-279 branch. Add debug statements for AM not launching
  1318. (mahadev)
  1319. MAPREDUCE-279. Fix in MR-279 branch. Fixing race condition leader to hung jobs
  1320. in scheduler negotiator (mahadev)
  1321. MAPREDUCE-279. Fix in MR-279 branch. Add debug config for delaying delete of
  1322. local files. (cdouglas)
  1323. MAPREDUCE-2527. Metrics for MRAppMaster (Luke lu via mahadev)
  1324. MAPREDUCE-2532. Metrics for NodeManager (Luke Lu via mahadev)
  1325. MAPREDUCE-279. Fix in MR-279 branch. Fixed an NPE during handling of unnecessary
  1326. reservations in CS. (acmurthy)
  1327. MAPREDUCE-279. Fix in MR-279 branch. Fix for public dist cache to work with non
  1328. default hdfs (mahadev &ddas)
  1329. MAPREDUCE-279. Fix in MR-279 branch. Making streaming -file option work. Also
  1330. minor fixes for successful compilation of contrib tests. (vinodkv)
  1331. MAPREDUCE-2536. Backporting changes to MR-279.
  1332. MAPREDUCE-279. Fix in MR-279 branch. Bugfix for using user staging directory for
  1333. history files (Siddharth Seth via mahadev)
  1334. MAPREDUCE-279. Fix in MR-279 branch. First fix for making basic speculative
  1335. execution work (ddas)
  1336. MAPREDUCE-279. Fix in MR-279 branch. Fixes for TestFail/Kill (ddas)
  1337. MAPREDUCE-279. Fix in MR-279 branch. Set correct version of avro-maven-plugin
  1338. that is available in apache maven repositories. (vinodkv)
  1339. MAPREDUCE-279. Fix in MR-279 branch. Fixing and reneabling
  1340. TestContainerTokenSecretManager. (vinodkv)
  1341. MAPREDUCE-279. Fix in MR-279 branch. Cleaning up configuration constants in
  1342. mapreduce modules. (vinodkv)
  1343. MAPREDUCE-279. Fix in MR-279 branch. Fixing NPE on speculator in MRAppMaster and
  1344. making job-history optional in tests to make test goal succeed. (vinodk and
  1345. sharadag).
  1346. MAPREDUCE-279. Fix in MR-279 branch. Fixed NPE in CS by checking Application
  1347. state before scheduling and fixing synchronization in CS. (acmurthy)
  1348. MAPREDUCE-279. Fix in MR-279 branch. Making pipes work with YARN. Changed pipes
  1349. to get log-locations from an environmental variable. (vinodkv)
  1350. MAPREDUCE-279. Fix in MR-279 branch. Ensure 'lost' NodeManagers are dealt
  1351. appropriately, the containers are released correctly. (acmurthy)
  1352. MAPREDUCE-279. Fix in MR-279 branch. Adding some more logging for AM expiry logs
  1353. (mahadev)
  1354. MAPREDUCE-279. Fix in MR-279 branch. Reduce ramp up and zero maps support.
  1355. (sharad)
  1356. MAPREDUCE-279. Fix in MR-279 branch. Allowing hdfs calls from streaming/pipes
  1357. tasks. (vinodkv)
  1358. MAPREDUCE-279. Fix in MR-279 branch. Added ability to decommission nodes and
  1359. completed RM administration tools to achieve parity with JobTracker. (acmurthy)
  1360. MAPREDUCE-2551. Added JobSummaryLog. (Siddharth Seth via acmurthy)
  1361. MAPREDUCE-2552. Fixed NPE in CompletedJob in JobHistoryServer. (Siddharth Seth
  1362. via acmurthy)
  1363. MAPREDUCE-279. Fix in MR-279 branch. Fix reduce slow start. (sharad)
  1364. MAPREDUCE-279. Fix in MR-279 branch. Fixed TestFifoScheduler. (acmurthy)
  1365. MAPREDUCE-279. Fix in MR-279 branch. Fix build issue for using yarn.version
  1366. instead of hadoop-mapred.version (mahadev and giri)
  1367. MAPREDUCE-279. Fix in MR-279 branch. Fixes in the handling of KILL events in the
  1368. SUCCEEDED state for tasks in the application master (ddas)
  1369. MAPREDUCE-279. Fix in MR-279 branch. Fix for NPE in TestRMNMRPCResponseId.
  1370. (mahadev)
  1371. MAPREDUCE-279. Fix in MR-279 branch. Fix a deadlock in the resourcemanager.
  1372. (mahadev)
  1373. MAPREDUCE-279. Fix in MR-279 branch. NodeStatus.getNodeHealthStatus().setBlah
  1374. broken (Siddharth Seth)
  1375. MAPREDUCE-279. Fix in MR-279 branch. Fix another NPE in TestRMNMRPCResponseId.
  1376. (mahadev)
  1377. MAPREDUCE-279. Fix in MR-279 branch. Fix for NPE in TestNMExpiry (siddharth
  1378. seth)
  1379. MAPREDUCE-279. Fix in MR-279 branch. Making each node aggregate all its
  1380. user-logs to a separate hdfs file. (vinodkv)
  1381. MAPREDUCE-279. Fix in MR-279 branch. Fix calculation of max-capacity for a
  1382. queue, also fixed a bug in registration of NodeManagers. (acmurthy)
  1383. MAPREDUCE-279. Fix in MR-279 branch. More cleaning up constants, removing stale
  1384. code, and making conspicuous the envs that apps depend on to be provided by
  1385. YARN. (vinodkv)
  1386. MAPREDUCE-279. Fix in MR-279 branch. Fix container size rounding in AM and
  1387. headroom in RM. (acmurthy and sharad)
  1388. MAPREDUCE-279. Fix in MR-279 branch. Disable Job acls until fixed (mahadev)
  1389. MAPREDUCE-279. Fix in MR-279 branch. Fix to report job status if the application
  1390. is KILLED/FAILED. (mahadev)
  1391. MAPREDUCE-279. Fix in MR-279 branch. Fix a corner case in headroom computation -
  1392. now reservations are taken into account and headroom is computed much later to
  1393. account for allocations/reservations. (acmurthy)
  1394. MAPREDUCE-2537. The RM writes its log to
  1395. yarn-mapred-resourcemanager-<RM_Host>.out (Robert Evans via mahadev)
  1396. MAPREDUCE-279. Fix in MR-279 branch. Fix logging for showing the state of job
  1397. (FAILED/KILLED/SUCCEEDED) when it completes (mahadev)
  1398. MAPREDUCE-279. Fix in MR-279 branch. Re-enabled TestCapacityScheduler.
  1399. (acmurthy)
  1400. MAPREDUCE-279. Fix in MR-279 branch. Support for min and max container capacity.
  1401. (acmurthy and sharad)
  1402. MAPREDUCE-2531. Fixed jobcontrol to downgrade JobID. (Robert Evans via acmurthy)
  1403. MAPREDUCE-2539. Fixed NPE in getMapTaskReports in JobClient. (Robert Evans via
  1404. acmurthy)
  1405. MAPREDUCE-279. Fix in MR-279 branch. Fixing the wrong config key used in
  1406. JobHistory that prevented configuring move-thread interval. (vinodkv)
  1407. MAPREDUCE-279. Fix in MR-279 branch. Fixed inconsistency in QueueACL enums.
  1408. (acmurthy)
  1409. MAPREDUCE-279. Fix in MR-279 branch. Fix various issues with Web UI's. (Luke Lu)
  1410. MAPREDUCE-279. Fix in MR-279 branch. Fix class cast exception in Task abort for
  1411. old mapreduce apis. (sharad)
  1412. MAPREDUCE-279. Fix in MR-279 branch. Add deletion of distributed cache
  1413. resources. (cdouglas)
  1414. MAPREDUCE-279. Fix in MR-279 branch. Disable aggregation of logs onto DFS till
  1415. JobHistoryServer starts serving logs. (vinodkv)
  1416. MAPREDUCE-279. Fix in MR-279 branch. Cleanup redundant code in TaskAttemptImpl.
  1417. (sharad)
  1418. MAPREDUCE-279. Fix in MR-279 branch. Work around broken signaling in public
  1419. cache. (cdouglas)
  1420. MAPREDUCE-2566. YarnConfiguration should reloadConfiguration if instantiated
  1421. with a non YarnConfiguration object. (Siddharth Seth)
  1422. MAPREDUCE-279. Fix in MR-279 branch. Fully resolve paths when launching
  1423. containers. (Siddharth Seth)
  1424. MAPREDUCE-279. Fix in MR-279 branch. Re-enabling Uber-AM feature. (vinodkv)
  1425. MAPREDUCE-279. Fix in MR-279 branch. Fixed deadlock during expiring NMs.
  1426. (acmurthy)
  1427. MAPREDUCE-279. Fix in MR-279 branch. Solving NPEs during
  1428. ContainerManager#StopContainer. Also removing the unused
  1429. ContainerManager#CleanupContainer api. (vinodkv)
  1430. MAPREDUCE-279. Fix in MR-279 branch. Remove retries in dist cache so that NM's
  1431. do not shutdown (mahadev)
  1432. MAPREDUCE-279. Fix in MR-279 branch. Fix classpath construction for Task.
  1433. (vinodkv via sharad)
  1434. MAPREDUCE-279. Fix in MR-279 branch. Sending Job diagnostics from AM to RM and
  1435. redirect to history-url on job completion. (vinodkv and sharadag)
  1436. MAPREDUCE-279. Fix in MR-279 branch. Added clover in pom dependency. (sharad)
  1437. MAPREDUCE-279. Fix in MR-279 branch. Intermittent TestMRApp failures on faster
  1438. Linux desktop. (Luke lu via mahadev)
  1439. MAPREDUCE-279. Fix in MR-279 branch. Including source files in release
  1440. distribution (Luke Lu via mahadev)
  1441. MAPREDUCE-279. Fix in MR-279 branch. Intermittent TestMRApp failures on faster
  1442. Linux desktop (part 2) (Luke lu via mahadev)
  1443. MAPREDUCE-279. Fix in MR-279 branch. Disable Uber AM. (mahadev)
  1444. MAPREDUCE-279. Fix in MR-279 branch. Added few job diagnostic messages. (sharad)
  1445. MAPREDUCE-279. Fix in MR-279 branch. Add ability to includes src files in
  1446. assembly target for maven (Luke Lu via mahadev)
  1447. MAPREDUCE-2582. Cleanup JobHistory event generation.(Siddharth Seth via sharad)
  1448. MAPREDUCE-279. Fix in MR-279 branch. Fix rounding off problem in reduce ramp up.
  1449. (sharad)
  1450. MAPREDUCE-279. Fix in MR-279 branch. Fix more rounding off problems in reduce
  1451. ramp up. Also fix a bug preventing the application of the cap on reduce ramp-up.
  1452. (Sharad Agarwal via vinodkv)
  1453. MAPREDUCE-279. Fix in MR-279 branch. Fix to exclude images dir into the tar
  1454. distribution (Luke Lu via gkesavan)
  1455. MAPREDUCE-279. Fix in MR-279 branch. Changes a couple of usages of FileContext
  1456. to FileSystem in YarnRunner to handle distributed cache path resolutions on
  1457. non-default filesystems. (ddas)
  1458. MAPREDUCE-279. Fix in MR-279 branch. Display failed/killed attempts of the task
  1459. on MR AM UI separately. (vinodkv)
  1460. MAPREDUCE-279. Fix in MR-279 branch. Miscellaneous UI fixes + source code
  1461. formatting for MR JobHistoryEventHandler. (vinodkv)
  1462. MAPREDUCE-279. Fix in MR-279 branch. Fixing broken link to logs for container on
  1463. NM web UI. (vinodkv)
  1464. MAPREDUCE-279. Fix in MR-279 branch. Fixing the bug which was causing FAILED
  1465. jobs to be displayed as COMPLETED on the RM UI. (vinodkv)
  1466. MAPREDUCE-279. Fix in MR-279 branch. Job level node blacklisting. (sharad)
  1467. MAPREDUCE-279. Fix in MR-279 branch. Fix NPE in history event handling
  1468. (Siddharth Seth via mahadev)
  1469. MAPREDUCE-2569. Ensure root queue allocated 100% capacity. (Jonathan Eagles via
  1470. cdouglas)
  1471. MAPREDUCE-279. Fix in MR-279 branch. Fix ClassCastException in JobHistoryServer
  1472. for certain jobs. (Siddharth Seth via llu)
  1473. MAPREDUCE-279. Fix in MR-279 branch. Changes for invoking rack resolution in the
  1474. RM and in the AM (ddas)
  1475. MAPREDUCE-279. Fix in MR-279 branch. Fix concurrent modification exception in
  1476. the Capacity Scheduler (mahadev)
  1477. MAPREDUCE-279. Fix in MR-279 branch. Fix stopContainer for setsid challenged
  1478. platforms. (llu)
  1479. MAPREDUCE-2587. Generate yarn version for UI. (Thomas Graves via lluts page to
  1480. the history server UI. (vinodkv)
  1481. MAPREDUCE-279. Fix in MR-279 branch. Bug fix to set correct state on containers
  1482. so as to avoid duplicate containers from RM to AM. (vinodkv)
  1483. MAPREDUCE-279. Fix in MR-279 branch. Hack until MAPREDUCE-2365 is fixed to make
  1484. PIG work with MRV2. (mahadev)
  1485. MAPREDUCE-279. Fix in MR-279 branch. Changes a couple of usages of FileContext
  1486. to FileSystem in TaskAttemptImpl to handle distributed cache path resolutions on
  1487. non-default filesystems. (ddas)
  1488. MAPREDUCE-279. Fix in MR-279 branch. Fix NPE when requesting attempts for
  1489. completed jobs. (Siddharth Seth via llu)
  1490. MAPREDUCE-279. Fix in MR-279 branch. Improve logging for AM when requesting
  1491. containers to show the right ask and release fields (mahadev)
  1492. MAPREDUCE-279. Fix in MR-279 branch. Fix race condition between multiple
  1493. localizers on a single node. (cdouglas via mahadev)
  1494. MAPREDUCE-279. Fix in MR-279 branch. Fix RM app start/finish time and
  1495. diagnostics. (llu)
  1496. MAPREDUCE-279. Fix in MR-279 branch. Fix to schedule reduces irrespective of the
  1497. headroom when all maps are done so as to avoid stall in reduce-scheduling when
  1498. slow-start is disabled. (Sharad Agarwal via vinodkv).
  1499. MAPREDUCE-279. Fix in MR-279 branch. Disabling locality-wait in
  1500. CapacityScheduler for now to prevent uber-slow scheduling for apps with no
  1501. data-locality constraints (sleep-job like). (vinodkv)
  1502. MAPREDUCE-279. Fix in MR-279 branch. Fixing scheduling deadlock in AM because of
  1503. incorrect headRoom values from RM. The bug happens when AM releases containers
  1504. and RM decrements current memory usage twice for all those containers. (vinodkv)
  1505. MAPREDUCE-2611. Fix counters, finish times etc. in job history. (Siddharth Seth
  1506. via llu)
  1507. MAPREDUCE-279. Fix in MR-279 branch. Fix for ConcurrentModification exception
  1508. while iterating through tokens in a UGI in ContainerLauncherImpl. (ddas)
  1509. MAPREDUCE-279. Fix in MR-279 branch. Fix for NPE in YarnChild that was causing
  1510. lots of tasks to fail. (vinodkv)
  1511. MAPREDUCE-2615. Make killJob go through AM and fix JobSummaryLog. (Siddharth
  1512. Seth via llu)
  1513. MAPREDUCE-279. Fix in MR-279 branch. Fix class cast exception in release
  1514. reserved containers in capacity scheduler (mahadev)
  1515. MAPREDUCE-279. Fix in MR-279 branch. Fix diagnostics display for more than 100
  1516. apps in RM. (llu)
  1517. MAPREDUCE-279. Fix in MR-279 branch. Fix some invalid transitions in the RM.
  1518. (vinodkv via ddas)
  1519. MAPREDUCE-2618. Fix NPE in 0 map 0 reduce jobs. (Jeffrey Naisbitt via llu)
  1520. MAPREDUCE-2625. Add version info to nodemanager info page. (Jonathan Eagles via
  1521. llu)
  1522. MAPREDUCE-279. Fix in MR-279 branch. (1) Faster retries from AM to HistoryServer
  1523. (2) Correct diagnostics for containers. (vinodkv)
  1524. MAPREDUCE-279. Fix in MR-279 branch. Major ASM cleanup. Streamlining classes,
  1525. interface and events. (vinodkv)
  1526. MAPREDUCE-279. Fix in MR-279 branch. Reinstate resolve path fixes for viewfs.
  1527. (Siddharth Seth via llu)
  1528. MAPREDUCE-2633. Add a getCounter(Enum) method to the Counters record. (Josh
  1529. Wills via sharad)
  1530. MAPREDUCE-2645. Updates to MRv2 INSTALL and README documentation. (Josh Wills
  1531. via vinodkv)
  1532. MAPREDUCE-2628. Add compiled on date to NM and RM info/about page.
  1533. MAPREDUCE-2400. Remove Cluster's dependency on JobTracker via a ServiceProvider
  1534. for the actual implementation. (tomwhite via acmurthy)
  1535. MAPREDUCE-2663. Refactoring StateMachineFactory inner classes. (ahmed radwan via
  1536. mahadev)
  1537. MAPREDUCE-2678. minimum-user-limit-percent no longer honored. (naisbitt via
  1538. mahadev)
  1539. MAPREDUCE-2630. refreshQueues leads to NPEs when used w/FifoScheduler. (Josh
  1540. Wills via mahadev)
  1541. MAPREDUCE-2644. NodeManager fails to create containers when NM_LOG_DIR is not
  1542. explicitly set in the Configuration. (Josh Wills via vinodkv)
  1543. MAPREDUCE-2661. Fix TaskImpl to not access MapTaskImpl. (Ahmed Radwan via
  1544. sharad)
  1545. HADOOP-6929. Backport changes to MR-279 (mahadev and owen)
  1546. HADOOP-6929. Making Security Info abstract and not an interface (mahadev)
  1547. MAPREDUCE-2667. mapred job -kill leaves application in RUNNING state (thomas
  1548. graves via mahadev)
  1549. MAPREDUCE-2664. Implement JobCounters for Mtions as asynchronous. (vinodkv,
  1550. sharad and acmurthy)
  1551. MAPREDUCE-2773. server.api.records.NodeHealthStatus renamed but not updated in
  1552. client NodeHealthStatus.java (Thomas Graves via mahadev)
  1553. MAPREDUCE-2772. Fix MR-279 build after common mavenization. (Robert Joseph Evans
  1554. via llu)
  1555. MAPREDUCE-2772. Fix MR-279 build after common mavenization, part 2. (Thomas
  1556. Graves via llu)
  1557. MAPREDUCE-279. Fix in MR-279 branch. Harmonize slf4j versions. (llu)
  1558. MAPREDUCE-279. Fix in MR-279 branch. Fix NPE in FifoScheduler. (mahadev)
  1559. MAPREDUCE-2776. Fix some of the yarn findbug warnings. (Siddharth Seth via
  1560. mahadev)
  1561. MAPREDUCE-279. Fix in MR-279 branch. Fix findbugs warnings in mr-client modules,
  1562. part 1 (mahadev)
  1563. MAPREDUCE-279. Fix in MR-279 branch. Fix findbugs warnings in mr-client modules
  1564. part 2 (mahadev)
  1565. MAPREDUCE-279. Fix in MR-279 branch. Fix findbugs warnings in mr-client modules
  1566. part 3 (mahadev)
  1567. MAPREDUCE-279. Fix in MR-279 branch. Fix the poms to enable 0.23 snapshots for
  1568. hdfs/common from apache nightly builds (gkesavan)
  1569. MAPREDUCE-279. Fix in MR-279 branch. Fix ivy conf to work with the hadoop common
  1570. trunk maven build changes. (Giridharan Kesavan)
  1571. MAPREDUCE-279. Fix in MR-279 branch. Patch for findbugs warnings in Resource
  1572. Manager (Siddharth Seth via mahadev)
  1573. MAPREDUCE-279. Fix in MR-279 branch. Fix for running ant targets to use the
  1574. right set of common/test jars (gkesavan via mahadev)
  1575. MAPREDUCE-2782. Unit tests for CapacityScheduler. (acmurthy)
  1576. MAPREDUCE-2706. Log job submission failures. (Jeffrey Naisbitt via acmurthy)
  1577. MAPREDUCE-2781. mr279 RM application finishtime not set (Thomas Graves via
  1578. mahadev)
  1579. MAPREDUCE-279. Fix in MR-279 branch. Fixed CS locality wait factor. (acmurthy)
  1580. MAPREDUCE-2808. pull MAPREDUCE-2797 into mr279 branch (Thomas Graves via
  1581. mahadev)
  1582. MAPREDUCE-2639. Bug fixes in speculate.DataStatistics. (Josh Wills via acmurthy)
  1583. MAPREDUCE-2839. Fixed TokenCache to get delegation tokens using both new and old
  1584. apis. (Siddharth Seth via acmurthy)
  1585. MAPREDUCE-2727. Fix divide-by-zero error in SleepJob for sleepCount equals 0.
  1586. (Jeffrey Naisbitt via acmurthy)
  1587. MAPREDUCE-2839. Fixed TokenCache to get delegation tokens using both new
  1588. and old apis. (Siddharth Seth via acmurthy)
  1589. MAPREDUCE-2727. Fix divide-by-zero error in SleepJob for sleepCount equals
  1590. 0. (Jeffrey Naisbitt via acmurthy)
  1591. MAPREDUCE-2860. Fix log4j logging in the maven test cases. (mahadev)
  1592. MAPREDUCE-2867. Remove Unused TestApplicaitonCleanup in resourcemanager/applicationsmanager.
  1593. (mahadev)
  1594. MAPREDUCE-2868. ant build broken in hadoop-mapreduce dir (mahadev, giri and arun via mahadev)
  1595. MAPREDUCE-2649. Handling of finished applications in RM. (Thomas Graves
  1596. via acmurthy)
  1597. MAPREDUCE-2838. Fix MapReduce builds to use new hadoop-common test jars.
  1598. (gkesavan via acmurthy)
  1599. MAPREDUCE-2859. Fix eclipse plugin contrib module compilation (gkesavan)
  1600. MAPREDUCE-2846. Fix missing synchronization in the task log management.
  1601. (omalley)
  1602. MAPREDUCE-2807. Fix AM restart and client redirection. (sharad)
  1603. MAPREDUCE-2877. Add missing Apache license header in some files in MR
  1604. and also add the rat plugin to the poms. (mahadev)
  1605. MAPREDUCE-2796. Set start times for MR applications for clients to see.
  1606. (Devaraj K via acmurthy)
  1607. MAPREDUCE-2879. Fix version for MR-279 to 0.23.0. (acmurthy)
  1608. MAPREDUCE-2881. Fix to include log4j 1.2.16 depenency (gkesavan)
  1609. MAPREDUCE-2885. Fix mapred-config.sh to look for hadoop-config.sh in
  1610. HADOOP_COMMON_HOME/libexec. (acmurthy)
  1611. MAPREDUCE-2893. Remove duplicate entry of YarnClientProtocolProvider in
  1612. ClientProtocolProvider services file. (Liang-Chi Hsieh via acmurthy)
  1613. MAPREDUCE-2891. Javadoc for AMRMProtocol and related records. (acmurthy)
  1614. MAPREDUCE-2898. Javadoc for ContainerManager protocol and related records.
  1615. (acmurthy)
  1616. MAPREDUCE-2904. Fixed bin/yarn to correctly include HDFS jars and
  1617. clean up of stale refs to pre-mavenized Hadoop Common and HDFS.
  1618. (Sharad Agarwal and Arun C. Murthy via acmurthy)
  1619. MAPREDUCE-2737. Update the progress of jobs on client side. (Siddharth Seth
  1620. and Mahadev Konar via mahadev)
  1621. MAPREDUCE-2886. Fix Javadoc warnings in MapReduce. (mahadev)
  1622. MAPREDUCE-2897. Javadoc for ClientRMProtocol protocol and related records.
  1623. (acmurthy)
  1624. MAPREDUCE-2916. Ivy build for MRv1 fails with bad organization for
  1625. common daemon. (mahadev)
  1626. MAPREDUCE-2917. Fixed corner case in container reservation which led to
  1627. starvation and hung jobs. (acmurthy)
  1628. MAPREDUCE-2756. Better error handling in JobControl for failed jobs.
  1629. (Robert Evans via acmurthy)
  1630. MAPREDUCE-2716. MRReliabilityTest job fails because of missing
  1631. job-file. (Jeffrey Naisbitt via vinodkv)
  1632. MAPREDUCE-2882. TestLineRecordReader depends on ant jars. (todd)
  1633. MAPREDUCE-2687. Fix NodeManager to use the right version of
  1634. LocalDirAllocator.getLocalPathToWrite. (mahadev & acmurthy)
  1635. MAPREDUCE-2800. Set final progress for tasks to ensure all task information
  1636. is correctly logged to JobHistory. (Siddharth Seth via acmurthy)
  1637. MAPREDUCE-2938. Log application submission failure in CapacityScheduler.
  1638. (acmurthy)
  1639. MAPREDUCE-2948. Hadoop streaming test failure, post MR-2767 (mahadev)
  1640. MAPREDUCE-2908. Fix all findbugs warnings. (vinodkv via acmurthy)
  1641. MAPREDUCE-2947. Fixed race condition in AuxiliaryServices. (vinodkv via
  1642. acmurthy)
  1643. MAPREDUCE-2844. Fixed display of nodes in UI. (Ravi Teja Ch N V via
  1644. acmurthy)
  1645. MAPREDUCE-2677. Fixed 404 for some links from HistoryServer. (Robert Evans
  1646. via acmurthy)
  1647. MAPREDUCE-2937. Ensure reason for application failure is displayed to the
  1648. user. (mahadev via acmurthy)
  1649. MAPREDUCE-2953. Fix a race condition on submission which caused client to
  1650. incorrectly assume application was gone by making submission synchronous
  1651. for RMAppManager. (Thomas Graves via acmurthy)
  1652. MAPREDUCE-2963. Fix hang in TestMRJobs. (Siddharth Seth via acmurthy)
  1653. MAPREDUCE-2954. Fixed a deadlock in NM caused due to wrong synchronization
  1654. in protocol buffer records. (Siddharth Seth via vinodkv)
  1655. MAPREDUCE-2975. Fixed YARNRunner to use YarnConfiguration rather than
  1656. Configuration. (mahadev via acmurthy)
  1657. MAPREDUCE-2971. ant build mapreduce fails protected access jc.displayJobList
  1658. (jobs) (Thomas Graves via mahadev)
  1659. MAPREDUCE-2691. Finishing up the cleanup of distributed cache file resources
  1660. and related tests. (Siddharth Seth via vinodkv)
  1661. MAPREDUCE-2749. Ensure NM registers with RM after starting all its services
  1662. correctly. (Thomas Graves via acmurthy)
  1663. MAPREDUCE-2979. Removed the needless ClientProtocolProvider configuration
  1664. from the hadoop-mapreduce-client-core module. (Siddharth Seth via vinodkv)
  1665. MAPREDUCE-2985. Fixed findbugs warnings in ResourceLocalizationService.
  1666. (Thomas Graves via acmurthy)
  1667. MAPREDUCE-2874. Fix formatting of ApplicationId in web-ui. (Eric Payne via
  1668. acmurthy)
  1669. MAPREDUCE-2995. Better handling of expired containers in MapReduce
  1670. ApplicationMaster. (vinodkv via acmurthy)
  1671. MAPREDUCE-2995. Fixed race condition in ContainerLauncher. (vinodkv via
  1672. acmurthy)
  1673. MAPREDUCE-2949. Fixed NodeManager to shut-down correctly if a service
  1674. startup fails. (Ravi Teja via vinodkv)
  1675. MAPREDUCE-3005. Fix both FifoScheduler and CapacityScheduler to correctly
  1676. enforce locality constraints. (acmurthy)
  1677. MAPREDUCE-3007. Fixed Yarn Mapreduce client to be able to connect to
  1678. JobHistoryServer in secure mode. (vinodkv)
  1679. MAPREDUCE-2987. Fixed display of logged user on RM Web-UI. (Thomas Graves
  1680. via acmurthy)
  1681. MAPREDUCE-3006. Fixed MapReduce AM to exit only after properly writing out
  1682. history file. (vinodkv)
  1683. MAPREDUCE-2925. Fixed Yarn+MR client code to behave saner with completed
  1684. jobs. (Devaraj K via vinodkv)
  1685. MAPREDUCE-3030. Fixed a bug in NodeId.equals() that was causing RM to
  1686. reject all NMs. (Devaraj K via vinodkv)
  1687. MAPREDUCE-3042. Fixed default ResourceTracker address. (Chris Riccomini
  1688. via acmurthy)
  1689. MAPREDUCE-3038. job history server not starting because conf() missing
  1690. HsController (Jeffrey Naisbitt via mahadev)
  1691. MAPREDUCE-3004. Fix ReduceTask to not assume 'local' mode in YARN. (Hitesh
  1692. Shah via acmurthy)
  1693. MAPREDUCE-3017. The Web UI shows FINISHED for killed/successful/failed jobs.
  1694. (mahadev)
  1695. MAPREDUCE-3040. Fixed extra copy of Configuration in
  1696. YarnClientProtocolProvider and ensured MiniMRYarnCluster sets JobHistory
  1697. configuration for tests. (acmurthy)
  1698. MAPREDUCE-3018. Fixed -file option for streaming. (mahadev via acmurthy)
  1699. MAPREDUCE-3036. Fixed metrics for reserved resources in CS. (Robert Evans
  1700. via acmurthy)
  1701. MAPREDUCE-2998. Fixed a bug in TaskAttemptImpl which caused it to fork
  1702. bin/mapred too many times. (vinodkv via acmurthy)
  1703. MAPREDUCE-3023. Fixed clients to display queue state correctly. (Ravi
  1704. Prakash via acmurthy)
  1705. MAPREDUCE-2970. Fixed NPEs in corner cases with different configurations
  1706. for mapreduce.framework.name. (Venu Gopala Rao via vinodkv)
  1707. MAPREDUCE-3062. Fixed default RMAdmin address. (Chris Riccomini
  1708. via acmurthy)
  1709. MAPREDUCE-3066. Fixed default ResourceTracker address for the NodeManager.
  1710. (Chris Riccomini via acmurthy)
  1711. MAPREDUCE-3044. Pipes jobs stuck without making progress. (mahadev)
  1712. MAPREDUCE-2754. Fixed MR AM stdout, stderr and syslog to redirect to
  1713. correct log-files. (Ravi Teja Ch N V via vinodkv)
  1714. MAPREDUCE-3073. Fixed build issues in MR1. (mahadev via acmurthy)
  1715. MAPREDUCE-2691. Increase threadpool size for launching containers in
  1716. MapReduce ApplicationMaster. (vinodkv via acmurthy)
  1717. MAPREDUCE-2990. Fixed display of NodeHealthStatus. (Subroto Sanyal via
  1718. acmurthy)
  1719. MAPREDUCE-3053. Better diagnostic message for unknown methods in ProtoBuf
  1720. RPCs. (vinodkv via acmurthy)
  1721. MAPREDUCE-2952. Fixed ResourceManager/MR-client to consume diagnostics
  1722. for AM failures in a couple of corner cases. (Arun C Murthy via vinodkv)
  1723. MAPREDUCE-3064. 27 unit test failures with Invalid
  1724. "mapreduce.jobtracker.address" configuration value for
  1725. JobTracker: "local" (Venu Gopala Rao via mahadev)
  1726. MAPREDUCE-3090. Fix MR AM to use ApplicationAttemptId rather than
  1727. (ApplicationId, startCount) consistently. (acmurthy)
  1728. MAPREDUCE-2646. Fixed AMRMProtocol to return containers based on
  1729. priority. (Sharad Agarwal and Arun C Murthy via vinodkv)
  1730. MAPREDUCE-3031. Proper handling of killed containers to prevent stuck
  1731. containers/AMs on an external kill signal. (Siddharth Seth via vinodkv)
  1732. MAPREDUCE-2984. Better error message for displaying completed containers.
  1733. (Devaraj K via acmurthy)
  1734. MAPREDUCE-3071. app master configuration web UI link under the Job menu
  1735. opens up application menu. (thomas graves via mahadev)
  1736. MAPREDUCE-3067. Ensure exit-code is set correctly for containers. (Hitesh
  1737. Shah via acmurthy)
  1738. MAPREDUCE-2999. Fix YARN webapp framework to properly filter servlet
  1739. paths. (Thomas Graves via vinodkv)
  1740. MAPREDUCE-3095. fairscheduler ivy including wrong version for hdfs.
  1741. (John George via mahadev)
  1742. MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev)
  1743. MAPREDUCE-3021. Change base urls for RM web-ui. (Thomas Graves via
  1744. acmurthy)
  1745. MAPREDUCE-3041. Fixed ClientRMProtocol to provide min/max resource
  1746. capabilities along-with new ApplicationId for application submission.
  1747. (Hitesh Shah via acmurthy)
  1748. MAPREDUCE-2843. Fixed the node-table to be completely displayed and making
  1749. node entries on RM UI to be sortable. (Abhijit Suresh Shingate via vinodkv)
  1750. MAPREDUCE-3110. Fixed TestRPC failure. (vinodkv)
  1751. MAPREDUCE-3078. Ensure MapReduce AM reports progress correctly for
  1752. displaying on the RM Web-UI. (vinodkv via acmurthy)
  1753. MAPREDUCE-3114. Fixed invalid ApplicationURL on RM WebUI. (Subroto Sanyal
  1754. via vinodkv)
  1755. MAPREDUCE-2791. Added missing info on 'job -status' output. (Devaraj K via
  1756. acmurthy)
  1757. MAPREDUCE-2996. Add uber-ness information to JobHistory. (Jonathan Eagles
  1758. via acmurthy)
  1759. MAPREDUCE-3050. Add ability to get resource usage information for
  1760. applications and nodes. (Robert Evans via acmurthy)
  1761. MAPREDUCE-3113. Ensure bin/yarn and bin/yarn-daemon.sh identify the root
  1762. of the install properly. (Xie Xianshan via acmurthy)
  1763. MAPREDUCE-3137. Fix broken merge of MAPREDUCE-2179. (Hitesh Shah via
  1764. acmurthy)
  1765. MAPREDUCE-2792. Replace usage of node ip-addresses with hostnames.
  1766. (vinodkv via acmurthy)
  1767. MAPREDUCE-3112. Fixed recursive sourcing of HADOOP_OPTS environment
  1768. variable. (Eric Yang)
  1769. MAPREDUCE-3056. Changed the default staging directory to not include
  1770. user.name to prevent issues with non-secure mode. (Devaraj K via vinodkv)
  1771. MAPREDUCE-2913. Fixed TestMRJobs.testFailingMapper to assert the correct
  1772. TaskCompletionEventStatus. (Jonathan Eagles via vinodkv)
  1773. MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per
  1774. queue per user. (John George via mahadev)
  1775. MAPREDUCE-2783. Fixing RM web-UI to show no tracking-URL when AM
  1776. crashes. (Eric Payne via vinodkv)
  1777. MAPREDUCE-3141. Fix the broken MRAppMaster to work over YARN in security
  1778. mode.(vinodkv)
  1779. MAPREDUCE-2751. Modified NodeManager to stop leaving around local files
  1780. after application finishes. (Siddharth Seth via vinodkv)
  1781. MAPREDUCE-3033. Ensure Master interface pays attention to classic v/s yarn
  1782. frameworks. (Hitesh Shah via acmurthy)
  1783. MAPREDUCE-2802. Ensure JobHistory filenames have jobId. (Jonathan Eagles
  1784. via acmurthy)
  1785. MAPREDUCE-2876. Use a different config for ContainerAllocationExpirer.
  1786. (Anupam Seth via acmurthy)
  1787. MAPREDUCE-3153. Fix TestFileOutputCommitter which was broken by
  1788. MAPREDUCE-2702. (mahadev via acmurthy)
  1789. MAPREDUCE-3123. Fix NM to quote symlink names to escape special
  1790. characters. (Hitesh Shah via acmurthy)
  1791. MAPREDUCE-3154. Fix JobSubmitter to check for output specs before copying
  1792. job submission files to fail fast. (Abhijit Suresh Shingate via acmurthy)
  1793. MAPREDUCE-3158. Fix test failures in MRv1 due to default framework being
  1794. set to yarn. (Hitesh Shah via acmurthy)
  1795. MAPREDUCE-3167. container-executor is not being packaged with the assembly
  1796. target. (mahadev)
  1797. MAPREDUCE-3020. Fixed TaskAttemptImpl to log the correct node-address for
  1798. a finished Reduce task. (Chackaravarthy via vinodkv)
  1799. MAPREDUCE-2668. Fixed AuxServices to send a signal on application-finish
  1800. to all the services. (Thomas Graves via vinodkv)
  1801. MAPREDUCE-3126. Fixed a corner case in CapacityScheduler where headroom
  1802. wasn't updated on changes to cluster size. (acmurthy)
  1803. MAPREDUCE-3140. Fixed the invalid JobHistory URL for failed
  1804. applications. (Subroto Sanyal via vinodkv)
  1805. MAPREDUCE-3125. Modified TaskImpl to consider only non-failed, non-killed
  1806. task-attempts for obtaining task's progress. (Hitesh Shah via vinodkv)
  1807. MAPREDUCE-2666. Retrieve shuffle port number from JobHistory on MR AM
  1808. restart. (Jonathan Eagles via acmurthy)
  1809. MAPREDUCE-2789. Complete schedulingInfo on CLI. (Eric Payne via acmurthy)
  1810. MAPREDUCE-3170. Fixed job output commit for deep hierarchies. (Hitesh Shah
  1811. via acmurthy)
  1812. MAPREDUCE-3124. Fixed location of native libs i.e. libhadoop.so for
  1813. containers. (John George via acmurthy)
  1814. MAPREDUCE-3057. Job History Server goes of OutOfMemory with 1200 Jobs
  1815. and Heap Size set to 10 GB. (Eric Payne via mahadev)
  1816. MAPREDUCE-2840. mr279 TestUberAM.testSleepJob test fails. (jonathan eagles
  1817. via mahadev)
  1818. MAPREDUCE-3190. Ensure bin/yarn fails early with a clear error message
  1819. when HADOOP_COMMON_HOME or HADOOP_HDFS_HOME are not set. (todd & acmurthy
  1820. via acmurthy)
  1821. MAPREDUCE-3189. Add link decoration back to MR2's CSS. (Todd Lipcon via
  1822. mahadev)
  1823. MAPREDUCE-3127. Changed default value of yarn.resourcemanager.acl.enable
  1824. to true and added some more documentation. (acmurthy)
  1825. MAPREDUCE-3032. Fixed TaskAttemptImpl so that JobHistory can have error
  1826. information about failed tasks. (Devaraj K via vinodkv)
  1827. MAPREDUCE-3196. TestLinuxContainerExecutorWithMocks fails on Mac OSX.
  1828. (Arun Murthy via mahadev)
  1829. MAPREDUCE-3197. TestMRClientService failing on building clean checkout of
  1830. branch 0.23 (mahadev)
  1831. MAPREDUCE-2762. Cleanup MR staging directory on completion. (mahadev via
  1832. acmurthy)
  1833. MAPREDUCE-3165. Ensure logging options are set correctly for MR AM and
  1834. tasks. (todd via acmurthy)
  1835. MAPREDUCE-3203. Fix some javac warnings in MRAppMaster. (mahadev)
  1836. MAPREDUCE-3162. Separated application-init and container-init event types
  1837. in NodeManager's Application state machine. (Todd Lipcon via vinodkv)
  1838. MAPREDUCE-3176. Fixed ant mapreduce tests that are timing out because
  1839. of wrong framework name. (Hitesh Shah via vinodkv)
  1840. MAPREDUCE-3181. Fixed MapReduce runtime to load yarn-default.xml and
  1841. yarn-site.xml. (acmurthy)
  1842. MAPREDUCE-2788. Normalize resource requests in FifoScheduler
  1843. appropriately. (Ahmed Radwan via acmurthy)
  1844. MAPREDUCE-2693. Fix NPE in job-blacklisting. (Hitesh Shah via acmurthy)
  1845. MAPREDUCE-3208. Fix NPE task/container log appenders. (liangzhwa via
  1846. acmurthy)
  1847. MAPREDUCE-3212. Fix usage/help message for bin/yarn. (Bhallamudi Venkata
  1848. Siva Kamesh via acmurthy)
  1849. MAPREDUCE-3179. Ensure failed tests exit with right error code. (Jonathan
  1850. Eagles via acmurthy)
  1851. MAPREDUCE-3188. Ensure correct shutdown in services. (todd via acmurthy)
  1852. MAPREDUCE-3226. Fix shutdown of fetcher threads. (vinodkv via acmurthy)
  1853. MAPREDUCE-3070. Fix NodeManager to use ephemeral ports by default.
  1854. (Devaraj K via acmurthy)
  1855. MAPREDUCE-3242. Trunk compilation broken with bad interaction from
  1856. MAPREDUCE-3070 and MAPREDUCE-3239. (mahadev)
  1857. MAPREDUCE-3058. Fixed MR YarnChild to report failure when task throws an
  1858. error and thus prevent a hanging task and job. (vinodkv)
  1859. MAPREDUCE-3087. Fixed the mapreduce classpath to correctly include the
  1860. generated-classpath file needed for tests. (Ravi Prakash via vinodkv)
  1861. MAPREDUCE-3233. Fixed a bug in MR Job so as to be able to restart the
  1862. application on AM crash. (Mahadev Konar via vinodkv)
  1863. MAPREDUCE-3028. Added job-end notification support. (Ravi Prakash via
  1864. acmurthy)
  1865. MAPREDUCE-3249. Ensure shuffle-port is correctly used duringMR AM recovery.
  1866. (vinodkv via acmurthy)
  1867. MAPREDUCE-3252. Fix map tasks to not rewrite data an extra time when
  1868. map output fits in spill buffer. (todd)
  1869. MAPREDUCE-3159. Ensure DefaultContainerExecutor doesn't delete application
  1870. directories during app-init. (todd via acmurthy)
  1871. MAPREDUCE-2746. Yarn servers can't communicate with each other with
  1872. hadoop.security.authorization set to true (acmurthy via mahadev)
  1873. MAPREDUCE-2821. Added missing fields (resourcePerMap & resourcePerReduce)
  1874. to JobSummary logs. (mahadev via acmurthy)
  1875. MAPREDUCE-3253. Fixed ContextFactory to clone JobContext correctly.
  1876. (acmurthy)
  1877. MAPREDUCE-3263. Fixed the MAPREDUCE-3028 commit which broke MR1. (Hitesh
  1878. Shah via acmurthy)
  1879. MAPREDUCE-3269. Fixed log4j properties to correctly set logging options
  1880. for JobHistoryServer vis-a-vis JobSummary logs. (mahadev via acmurthy)
  1881. MAPREDUCE-3250. When AM restarts, client keeps reconnecting to the new AM
  1882. and prints a lots of logs. (vinodkv via mahadev)
  1883. MAPREDUCE-3254. Fixed streaming to set the job.jar by using the right
  1884. JobConf ctor. (acmurthy)
  1885. MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically.
  1886. (acmurthy via mahadev)
  1887. MAPREDUCE-3175. Add authorization to admin web-pages such as /stacks, /jmx
  1888. etc. (Jonathan Eagles via acmurthy)
  1889. MAPREDUCE-3257. Added authorization checks for the protocol between
  1890. ResourceManager and ApplicationMaster. (vinodkv via acmurthy)
  1891. MAPREDUCE-3259. Added java.library.path of NodeManager to
  1892. ContainerLocalizer in LinuxContainerExecutor. (Kihwal Lee via acmurthy)
  1893. MAPREDUCE-3279. Fixed TestJobHistoryParsing which assumed user name to be
  1894. mapred all the time. (Siddharth Seth via acmurthy)
  1895. MAPREDUCE-3240. Fixed NodeManager to be able to forcefully cleanup its
  1896. containers (process-trees) irrespective of whether the container succeeded,
  1897. or killed. Contributed by Hitesh Shah.
  1898. MAPREDUCE-3281. Fixed a bug in TestLinuxContainerExecutorWithMocks. (vinodkv)
  1899. MAPREDUCE-3228. Fixed MR AM to timeout RPCs to bad NodeManagers. (vinodkv
  1900. via acmurthy)
  1901. MAPREDUCE-3284. Moved JobQueueClient to hadoop-mapreduce-client-core.
  1902. (acmurthy)
  1903. MAPREDUCE-3282. bin/mapred job -list throws exception. (acmurthy via
  1904. mahadev)
  1905. MAPREDUCE-3186. User jobs are getting hanged if the Resource manager
  1906. process goes down and comes up while job is getting executed.
  1907. (Eric Payne via mahadev)
  1908. MAPREDUCE-3285. Tests on branch-0.23 failing (Siddharth Seth via mahadev)
  1909. MAPREDUCE-3258. Fixed AM & JobHistory web-ui to display counters properly.
  1910. (Siddharth Seth via acmurthy)
  1911. MAPREDUCE-3290. Fixed a NPE in ClientRMService. (acmurthy)
  1912. MAPREDUCE-3185. RM Web UI does not sort the columns in some cases.
  1913. (Jonathan Eagles via mahadev)
  1914. MAPREDUCE-3292. In secure mode job submission fails with Provider
  1915. org.apache.hadoop.mapreduce.security.token.JobTokenIndentifier$Renewer
  1916. not found. (mahadev)
  1917. MAPREDUCE-3296. Fixed the remaining nine FindBugs warnings. (vinodkv)
  1918. MAPREDUCE-2775. Fixed ResourceManager and NodeManager to force a
  1919. decommissioned node to shutdown. (Devaraj K via vinodkv)
  1920. MAPREDUCE-3304. Fixed intermittent test failure due to a race in
  1921. TestRMContainerAllocator#testBlackListedNodes. (Ravi Prakash via acmurthy)
  1922. MAPREDUCE-3306. Fixed a bug in NodeManager ApplicationImpl that was causing
  1923. NodeManager to crash. (vinodkv)
  1924. MAPREDUCE-3295. TestAMAuthorization failing on branch 0.23. (vinodkv via mahadev)
  1925. MAPREDUCE-3183. hadoop-assemblies/src/main/resources/assemblies/hadoop-mapreduce-dist.xml
  1926. missing license header. (Hitesh Shah via tucu).
  1927. MAPREDUCE-3003. Publish MR JARs to Maven snapshot repository. (tucu)
  1928. MAPREDUCE-3199. Fixed pom files to include correct log4j configuration for
  1929. tests. (vinodkv)
  1930. MAPREDUCE-3204. mvn site:site fails on MapReduce. (tucu)
  1931. MAPREDUCE-3248. Fixed log4j properties. (vinodkv via acmurthy)
  1932. MAPREDUCE-3256. Added authorization checks for the protocol between
  1933. NodeManager and ApplicationMaster. (vinodkv via acmurthy)
  1934. MAPREDUCE-3274. Fixed a race condition in MRAppMaster that was causing a
  1935. task-scheduling deadlock. (Robert Joseph Evans via vinodkv)
  1936. MAPREDUCE-3171 merge from trunk reverted changes from MAPREDUCE-2747 MAPREDUCE-3240.
  1937. MAPREDUCE-3313. Fixed initialization of ClusterMetrics which was failing
  1938. TestResourceTrackerService sometimes. (Hitesh Shah via vinodkv)
  1939. MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories
  1940. in distributed-cache. (Hitesh Shah via vinodkv)
  1941. MAPREDUCE-2696. Fixed NodeManager to cleanup logs in a thread when logs'
  1942. aggregation is not enabled. (Siddharth Seth via vinodkv)
  1943. MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle
  1944. a couple of events in failure states correctly. (Hitesh Shah and Siddharth
  1945. Seth via vinodkv)
  1946. MAPREDUCE-3035. Fixed MR JobHistory to ensure rack information is present.
  1947. (chakravarthy via acmurthy)
  1948. MAPREDUCE-3321. Disabled a few MR tests for 0.23. (Hitesh Shah via
  1949. acmurthy)
  1950. MAPREDUCE-3220. Fixed TestCombineOutputCollector. (Devaraj K via acmurthy)
  1951. MAPREDUCE-3103. Implement Job ACLs for MRAppMaster.
  1952. (mahadev)
  1953. MAPREDUCE-3241. [Rumen] Fix Rumen to ignore the AMStartedEvent. (amarrk)
  1954. MAPREDUCE-3166. [Rumen] Make Rumen use job history api instead of relying
  1955. on current history file name format. (Ravi Gummadi via amarrk)
  1956. MAPREDUCE-3157. [Rumen] Fix TraceBuilder to handle 0.20 history file
  1957. names also. (Ravi Gummadi via amarrk)
  1958. MAPREDUCE-3081. Fix vaidya startup script. (gkesavan via suhas).
  1959. MAPREDUCE-2764. Fix renewal of dfs delegation tokens. (Owen via jitendra)
  1960. MAPREDUCE-3192. Fix Javadoc warning in JobClient.java and Cluster.java.
  1961. (jitendra)
  1962. MAPREDUCE-3237. Move LocalJobRunner to hadoop-mapreduce-client-core.
  1963. (tomwhite via acmurthy)
  1964. MAPREDUCE-3316. Rebooted link is not working properly.
  1965. (Bhallamudi Venkata Siva Kamesh via mahadev)
  1966. MAPREDUCE-3317. Rumen TraceBuilder is emiting null as hostname.
  1967. (Ravi Gummadi via mahadev)
  1968. MAPREDUCE-3332. contrib/raid compile breaks due to changes in hdfs/protocol/datatransfer/
  1969. Sender#writeBlock related to checksum handling (Hitesh Shah via mahadev)
  1970. MAPREDUCE-3337. Added missing license headers. (acmurthy)
  1971. Release 0.22.1 - Unreleased
  1972. INCOMPATIBLE CHANGES
  1973. NEW FEATURES
  1974. IMPROVEMENTS
  1975. OPTIMIZATIONS
  1976. BUG FIXES
  1977. MAPREDUCE-3837. Job tracker is not able to recover jobs after crash.
  1978. (Mayank Bansal via shv)
  1979. Release 0.22.0 - 2011-11-29
  1980. INCOMPATIBLE CHANGES
  1981. MAPREDUCE-1866. Removes deprecated class
  1982. org.apache.hadoop.streaming.UTF8ByteArrayUtils. (amareshwari)
  1983. MAPREDUCE-1664. Changes the behaviour of the combination of job-acls
  1984. when they function together with queue-acls. (Ravi Gummadi via vinodkv)
  1985. MAPREDUCE-2994. Fixed a bug in ApplicationID parsing that affects RM
  1986. UI. (Devaraj K via vinodkv)
  1987. MAPREDUCE-1788. o.a.h.mapreduce.Job shouldn't make a copy of the JobConf.
  1988. (Arun Murthy via mahadev)
  1989. NEW FEATURES
  1990. MAPREDUCE-1804. Stress-test tool for HDFS introduced in HDFS-708.
  1991. (Joshua Harlow via shv)
  1992. MAPREDUCE-220. Collect cpu and memory statistics per task. (Scott Chen via
  1993. acmurthy)
  1994. MAPREDUCE-1970. Reed-Solomon code implementation for HDFS RAID.
  1995. (Scott Chen via dhruba)
  1996. MAPREDUCE-2169. Integrated Reed-Solomon code with RaidNode. (Ramkumar
  1997. Vadali via schen)
  1998. MAPREDUCE-2936. Contrib Raid compilation broken after HDFS-1620. (vinodkv)
  1999. IMPROVEMENTS
  2000. MAPREDUCE-2141. Add an "extra data" field to Task for use by Mesos. (matei)
  2001. MAPREDUCE-2140. Regenerate fair scheduler design doc PDF. (matei)
  2002. MAPREDUCE-1546. Redirect all job pages to corresponding history page
  2003. if job is not in JT memory. (Scott Chen via sharad)
  2004. MAPREDUCE-1092. Enable assertions for unit tests. (Eli Collins via
  2005. cdouglas)
  2006. MAPREDUCE-1680. Add a metric recording JobTracker heartbeats processed.
  2007. (Dick King via cdouglas)
  2008. MAPREDUCE-1761. FairScheduler allows separate configuration of node
  2009. and rack locality wait time (Scott Chen via dhruba)
  2010. MAPREDUCE-1539. authorization checks for inter-server protocol
  2011. (based on HADOOP-6600) (Boris Shkolnik via shv)
  2012. MAPREDUCE-1798. Names the configuration keys for the Kerberos
  2013. principals better. (Boris Shkolnik via ddas)
  2014. MAPREDUCE-1773. streaming doesn't support jobclient.output.filter.
  2015. (Amareshwari Sriramadasu via vinodkv)
  2016. MAPREDUCE-1785. Add streaming config option for not emitting the key.
  2017. (Eli Collins via sharad)
  2018. MAPREDUCE-572. If #link is missing from uri format of -cacheArchive
  2019. then streaming does not throw error. (Amareshwari Sriramadasu via
  2020. vinodkv)
  2021. MAPREDUCE-1545. Add timestamps for first task type launched in job summary.
  2022. (Luke Lu via cdouglas)
  2023. MAPREDUCE-1543. Add an audit log for authentication events. (Amar Kamat and
  2024. Luke Lu via cdouglas)
  2025. MAPREDUCE-1762. Add ability to set values of task counters. (Scott Chen via
  2026. cdouglas)
  2027. MAPREDUCE-1533. Reduce overhead of logging and string manipulation during
  2028. heartbeat processing. (Amar Kamat and Dick King via cdouglas)
  2029. MAPREDUCE-1516. JobTracker issues delegation tokens only if the user's
  2030. authentication is Kerberos. (Jitendra Pandey via ddas)
  2031. MAPREDUCE-647. Update distcp forrest documentation to reflect the changes
  2032. of HADOOP-5472, MAPREDUCE-642 and HADOOP-5620. (Rodrigo Schmidt via
  2033. szetszwo)
  2034. MAPREDUCE-1851. Documents configuration parameters in streaming.
  2035. (amareshwari)
  2036. MAPREDUCE-1868. Add a read and connection timeout to JobClient while
  2037. pulling tasklogs. (Krishna Ramachandran via acmurthy)
  2038. MAPREDUCE-1778. Ensure failure to setup CompletedJobStatusStore is not
  2039. silently ignored by the JobTracker. (Krishna Ramachandran via acmurthy)
  2040. MAPREDUCE-1850. Includes job submit host information (name and ip) in
  2041. jobconf and jobdetails display (Krishna Ramachandran via amareshwari)
  2042. MAPREDUCE-1893. Slive with multiple reducers. (shv)
  2043. MAPREDUCE-1248. Fixes redudant memory copying in StreamKeyValUtil.
  2044. (Ruibang He via amareshwari)
  2045. MAPREDUCE-1840. Enhancements to Gridmix benchmark simulating user
  2046. diversity, queue replay, and task duration for JobTracker load testing.
  2047. Also includes compatibility with security enhancements, and scalability
  2048. improvements. (Amar Kamat, Rahul Singh, Hong Tang, and cdouglas)
  2049. MAPREDUCE-1848. Put number of speculative, data local, rack local
  2050. tasks in JobTracker metrics. (Scott Chen via dhruba)
  2051. MAPREDUCE-1935. Makes the Distcp to work in a secure environment.
  2052. (Boris Shkolnik via ddas)
  2053. MAPREDUCE-1945. The MapReduce component for HADOOP-6632.
  2054. (Kan Zhang & Jitendra Pandey via ddas)
  2055. MAPREDUCE-1936. Modify Gridmix3 to support more tunable parameters for
  2056. stress submission and sleep jobs. (Hong Tang via cdouglas)
  2057. MAPREDUCE-1733. Makes pipes applications secure. (Jitendra Pandey via ddas)
  2058. MAPREDUCE-1566. Adds a configuration attribute using which job clients can
  2059. specify a credentials file. The tokens from there will be passed to the job.
  2060. (Jitendra Pandey and Owen O'Malley via ddas)
  2061. MAPREDUCE-1624. Documents the job credentials and associated details to do
  2062. with delegation tokens (on the client side).
  2063. (Jitendra Pandey and Devaraj Das via ddas)
  2064. MAPREDUCE-1834. TestSimulatorDeterministicReplay timesout on trunk.
  2065. (Hong Tang via mahadev)
  2066. MAPREDUCE-1993. Fixes test failure
  2067. TestTrackerDistributedCacheManagerWithLinuxTaskController. (Devaraj Das
  2068. via amareshwari)
  2069. MAPREDUCE-1523. Making Mumak work with Capacity-Scheduler (Anirban Das
  2070. via mahadev)
  2071. MAPREDUCE-1920. Enables completed jobstatus store by default. (Tom White
  2072. via amareshwari)
  2073. MAPREDUCE-1881. Improve TaskTrackerInstrumentation to enable collection of
  2074. advanced metrics. (Matei Zaharia via acmurthy)
  2075. MAPREDUCE-1548. Hadoop archives preserve times and other properties from
  2076. original files. (Rodrigo Schmidt via dhruba)
  2077. MAPREDUCE-1517. Supports streaming job to run in the background. (Bochun Bai
  2078. via amareshwari)
  2079. MAPREDUCE-1819. RaidNode is now smarter in submitting Raid jobs. (Ramkumar
  2080. Vadali via schen)
  2081. MAPREDUCE-2132. A command line option in RaidShell to fix blocks using raid
  2082. MAPREDUCE-2147. Remove redundant lines in JobInProgress ctor.
  2083. (Harsh J Chouraria via cutting)
  2084. HADOOP-7007. Update the hudson-test-patch ant target to work with the
  2085. latest test-patch.sh script (gkesavan)
  2086. MAPREDUCE-1818. RaidNode can specify scheduling parameters. (Ramkumar
  2087. Vadali via schen)
  2088. MAPREDUCE-2051. Contribute a fair scheduler preemption system test.
  2089. (Todd Lipcon via tomwhite)
  2090. MAPREDUCE-1892. RaidNode can allow layered policies more efficiently.
  2091. (Ramkumar Vadali via schen)
  2092. MAPREDUCE-1592. Generate Eclipse's .classpath file from Ivy config.
  2093. (tomwhite via nigel)
  2094. MAPREDUCE-2073. TestTrackerDistributedCacheManager should be up-front
  2095. about requirements on build environment. (Todd Lipcon via tomwhite)
  2096. MAPREDUCE-2093. Herriot JT and TT clients should vend statistics. (cos)
  2097. MAPREDUCE-2167. Faster directory traversal for raid node. (Ramkumar Vadali
  2098. via schen)
  2099. MAPREDUCE-1931. Gridmix forrest documentation . (Ranjit Mathew via vinodkv).
  2100. MAPREDUCE-2184. Port DistRaid.java to new mapreduce API. (Ramkumar Vadali
  2101. via schen)
  2102. MAPREDUCE-1878. Add MRUnit documentation. (Aaron Kimball via tomwhite)
  2103. MAPREDUCE-2180. Add coverage of fair scheduler servlet to system test (todd)
  2104. MAPREDUCE-2250. Fix logging in raid code. (Ramkumar Vadali via schen)
  2105. MAPREDUCE-2260. Remove auto-generated native build files. (rvs via eli)
  2106. MAPREDUCE-2314. configure files that are generated as part of the released
  2107. tarball need to have executable bit set (rvs via cos)
  2108. MAPREDUCE-1159. Limit Job name in web UI to be 80 char long. (Harsh J
  2109. Chouraria via szetszwo)
  2110. MAPREDUCE-2337. Remove dependence of public MapReduce API on classes in
  2111. server package. (tomwhite)
  2112. MAPREDUCE-2383. Improve documentation of DistributedCache methods (Harsh J
  2113. Chouraria via todd)
  2114. MAPREDUCE-2222. Ivy resolve force mode should be turned off by default.
  2115. (Luke Lu via tomwhite)
  2116. MAPREDUCE-2103. task-controller shouldn't require o-r permissions.
  2117. (todd via eli)
  2118. MAPREDUCE-2505. Explain how to use ACLs in the fair scheduler.
  2119. (matei via eli)
  2120. MAPREDUCE-3138. Add a utility to help applications bridge changes in
  2121. Context Objects APIs due to MAPREDUCE-954. (omalley via acmurthy)
  2122. OPTIMIZATIONS
  2123. MAPREDUCE-1354. Enhancements to JobTracker for better performance and
  2124. scalability. (Arun C. Murthy & Richard King via acmurthy)
  2125. MAPREDUCE-1829. JobInProgress.findSpeculativeTask should use min() to
  2126. find the candidate instead of sort(). (Scott Chen via vinodkv)
  2127. BUG FIXES
  2128. MAPREDUCE-1845. FairScheduler.tasksToPreempt() can return negative number.
  2129. (Scott Chen via matei)
  2130. MAPREDUCE-1707. TaskRunner can get NPE in getting ugi from TaskTracker.
  2131. (Vinod Kumar Vavilapalli)
  2132. MAPREDUCE-1532. Ensures that delegation tokens is obtained as the
  2133. actual user when the proxy-user is used for submitting jobs. Also
  2134. refactors the DelegationTokenToRenew class. (ddas)
  2135. MAPREDUCE-1558. Fixes MRAdmin to look up the principal of the
  2136. JobTracker and use that in the RefreshUserToGroupsMapping protocol and
  2137. RefreshAuthorizationPolicyProtocol. (Boris Shkolnik via ddas)
  2138. MAPREDUCE-1662. Remove unused methods from TaskRunner. (Amareshwari
  2139. Sriramadasu via cdouglas)
  2140. MAPREDUCE-1617. Use IPv4 stack for unit tests. (Amar Kamat and Luke Lu via
  2141. cdouglas)
  2142. MAPREDUCE-1599. Fixes MRBench so that it reuses tokens across jobs
  2143. correctly. (Jitendra Nath Pandey via ddas)
  2144. MAPREDUCE-1836. Refresh for proxy superuser config (mr part for HDFS-1096).
  2145. (Boris Shkolnik via shv)
  2146. MAPREDUCE-1505. Create RPC client on job submission, not in cstr of Job
  2147. instance. (Dick King via cdouglas)
  2148. MAPREDUCE-1813. NPE in PipeMapred.MRErrorThread. (Ravi Gummadi via vinodkv)
  2149. MAPREDUCE-1225. Fixes DistributedCache to check if the file is fresh or not,
  2150. for the first localization also. (Zhong Wang via amareshwari)
  2151. MAPREDUCE-1559. Fixes the token renewer to use the JobTracker's
  2152. credentials for talking to the NameNode. (ddas)
  2153. MAPREDUCE-1492. Delete obsolete har files used on the parity files
  2154. of hdfs raid. (Rodrigo Schmidt via dhruba)
  2155. MAPREDUCE-1857. Removes unused configuration parameters in streaming.
  2156. (amareshwari)
  2157. MAPREDUCE-1887. MRAsyncDiskService now properly absolutizes volume root
  2158. paths. (Aaron Kimball via zshao)
  2159. MAPREDUCE-1863. Fix NPE in Rumen when processing null CDF for failed task
  2160. attempts. (Amar Kamat via cdouglas)
  2161. MAPREDUCE-1864. Removes uninitialized/unused variables in
  2162. org.apache.hadoop.streaming.PipeMapRed. (amareshwari)
  2163. MAPREDUCE-1888. Fixes Streaming to override output key and value types,
  2164. only if mapper/reducer is a command. (Ravi Gummadi via amareshwari)
  2165. MAPREDUCE-577. Fixes duplicate records in StreamXmlRecordReader.
  2166. (Ravi Gummadi via amareshwari)
  2167. MAPREDUCE-1894. Fixed a bug in DistributedRaidFileSystem.readFully()
  2168. that was causing it to loop infinitely. (Ramkumar Vadali via dhruba)
  2169. MAPREDUCE-1838. Reduce the time needed for raiding a bunch of files
  2170. by randomly assigning files to map tasks. (Ramkumar Vadali via dhruba)
  2171. MAPREDUCE-1820. Fix InputSampler to clone sampled keys. (Alex Kozlov via
  2172. cdouglas)
  2173. MAPREDUCE-1528. Incorporates the changes to the credentials API done in
  2174. HADOOP-6845. Also, introduces Credentials in JobConf, and in JobContext.
  2175. (Jitendra Pandey and Arun Murthy via ddas)
  2176. MAPREDUCE-1865. Rumen should also support jobhistory files generated using
  2177. trunk. (Amar Kamat via amareshwari)
  2178. MAPREDUCE-1621. Fixes NPE in TextOutputReader.getLastOutput if it has never
  2179. read any output. (amareshwari)
  2180. MAPREDUCE-1911. Fixes errors in -info message in streaming. (amareshwari)
  2181. MAPREDUCE-1772. Corrects errors in streaming documentation in forrest.
  2182. (amareshwari)
  2183. MAPREDUCE-1925. Fix failing TestRumenJobTraces. (Ravi Gummadi via cdouglas)
  2184. MAPREDUCE-1718. Fixes a bug in the construction of jobconf key for the
  2185. mapping that the tasks use at runtime for looking up delegation tokens.
  2186. (Boris Shkolnik via ddas)
  2187. MAPREDUCE-1701. Fixes a problem to do with exception handling in
  2188. delegation-token renewals. (Boris Shkolnik via ddas)
  2189. MAPREDUCE-1686. Fixes StreamUtil.goodClassOrNull to find classes without
  2190. package names. (Paul Burkhardt via amareshwari)
  2191. MAPREDUCE-1288. Fixes TrackerDistributedCacheManager to take into account
  2192. the owner of the localized file in the mapping from cache URIs to
  2193. CacheStatus objects. (ddas)
  2194. MAPREDUCE-1982. Fixes Rumen's TraceBuilder to extract job name from either
  2195. of configuration properties "mapreduce.job.name" and "mapred.job.name".
  2196. (Ravi Gummadi via amareshwari)
  2197. MAPREDUCE-1958. The MapReduce part corresponding to the HADOOP-6873.
  2198. (Boris Shkolnik & Owen O'Malley via ddas)
  2199. MAPREDUCE-1900. TaskTracker and JobTracker closes FileSystems, opened on
  2200. behalf of users that it no longer requires. (Kan Zhang and ddas via ddas)
  2201. MAPREDUCE-1992. Fixes a problem to do with bringing up the JobTracker in
  2202. unsecure mode. (Kan Zhang via ddas)
  2203. MAPREDUCE-1999. Fixes ClientProtocol to use the correct
  2204. DelegationTokenSelector. (Jitendra Pandey via ddas)
  2205. MAPREDUCE-1780. AccessControlList.toString() is used for serialization of
  2206. ACL in JobStatus.java. (Ravi Gummadi via vinodkv)
  2207. MAPREDUCE-1961. Fix ConcurrentModificationException in Gridmix during
  2208. shutdown. (Hong Tang via cdouglas)
  2209. MAPREDUCE-2000. Fix parsing of JobHistory lines in Rumen when quotes are
  2210. escaped. (Hong Tang via cdouglas)
  2211. MAPREDUCE-2022. Fixes compilation errors in TestSubmitJob. (amareshwari)
  2212. MAPREDUCE-1670. RAID policies should not scan their own destination path.
  2213. (Ramkumar Vadali via dhruba)
  2214. MAPREDUCE-1668. RaidNode Hars a directory only if all its parity files
  2215. have been created. (Ramkumar Vadali via dhruba)
  2216. MAPREDUCE-2021. Fixes duplicate hostnames in CombineFileInputFormat's
  2217. split locations. (amareshwari)
  2218. MAPREDUCE-1375. Fixes flaky test TestFileArgs. (Todd Lipcon via
  2219. amareshwari)
  2220. MAPREDUCE-2023. TestDFSIO should not stop reading if curSize != bufferSize.
  2221. (Hong Tang via szetszwo)
  2222. MAPREDUCE-2031. Fixes test failures TestTaskLauncher and
  2223. TestTaskTrackerLocalization. (Ravi Gummadi via amareshwari)
  2224. MAPREDUCE-2046. Fixes CombineFileInputFormat to allow splits with size
  2225. less than DFS block size. (dhruba borthakur via amareshwari)
  2226. MAPREDUCE-1975. Fixes unnecessary InterruptedException log in gridmix.
  2227. (Ravi Gummadi via amareshwari)
  2228. MAPREDUCE-1597. Fixes CombineFileInputFormat to work with non-splittable
  2229. files. (amareshwari)
  2230. MAPREDUCE-2032. Fixes TestJobCleanup to cleanup test directory in
  2231. tearDown. (Dick King via amareshwari)
  2232. MAPREDUCE-1979. Fixes "Output directory already exists" error in gridmix
  2233. when gridmix.output.directory is not defined. (Ravi Gummadi via
  2234. amareshwari)
  2235. MAPREDUCE-1918. Adds documentation to Rumen. (Amar Kamat via amareshwari)
  2236. MAPREDUCE-2078. Fixes TraceBuilder to generate traces when a globbed job
  2237. history path is given. (Amar Kamat via amareshwari)
  2238. MAPREDUCE-1989. Fixes error message in gridmix when user resolver is set
  2239. and no user list is given. (Ravi Gummadi via amareshwari)
  2240. MAPREDUCE-2067. Distinct minicluster services (e.g. NN and JT) overwrite
  2241. each other's service policies. (Aaron T. Myers via tomwhite)
  2242. MAPREDUCE-2029. DistributedRaidFileSystem removes itself from FileSystem
  2243. cache when it is closed. (Ramkumar Vadali via dhruba)
  2244. MAPREDUCE-1816. HAR files used for RAID parity-bite have configurable
  2245. partfile size. (Ramkumar Vadali via dhruba)
  2246. MAPREDUCE-2082. Fixes Pipes to create the jobtoken file in the right
  2247. place. (Jitendra Pandey via ddas)
  2248. MAPREDUCE-2095. Fixes Gridmix to run from compressed traces. (Ranjit
  2249. Mathew via amareshwari)
  2250. MAPREDUCE-1908. DistributedRaidFileSystem now handles ChecksumException
  2251. correctly. (Ramkumar Vadali via schen)
  2252. MAPREDUCE-2126. JobQueueJobInProgressListener's javadoc is inconsistent
  2253. with source code. (Jingguo Yao via tomwhite)
  2254. MAPREDUCE-2143. HarFileSystem is able to handle spaces in pathnames.
  2255. (Ramkumar Vadali via dhruba)
  2256. MAPREDUCE-1867. Remove unused methods in
  2257. org.apache.hadoop.streaming.StreamUtil. (amareshwari via tomwhite)
  2258. MAPREDUCE-2146. Raid does not affect access time of a source file.
  2259. (Ramkumar Vadali via dhruba)
  2260. MAPREDUCE-2150. RaidNode periodically fixes corrupt blocks. (Ramkumar Vadali via
  2261. schen)
  2262. MAPREDUCE-2099. RaidNode recreates outdated parity HARs. (Ramkumar Vadali
  2263. via schen)
  2264. MAPREDUCE-2173. Fix race condition in TestBlockFixer that was
  2265. causing intermittent failure (Patrick Kling via dhruba)
  2266. MAPREDUCE-2142. Refactor RaidNode so that the map-reduce component is
  2267. clearly separated out. (Patrick Kling via dhruba)
  2268. MAPREDUCE-2179. Fix RaidBlockSender compilation failure. (Ramkumar Vadali
  2269. via schen)
  2270. MAPREDUCE-2034. TestSubmitJob triggers NPE instead of permissions error.
  2271. (Todd Lipcon via tomwhite)
  2272. MAPREDUCE-2195. New property for local conf directory in
  2273. system-test-mapreduce.xml file. (cos)
  2274. MAPREDUCE-1783. FairScheduler initializes tasks only when the job can be
  2275. run. (Ramkumar Vadali via schen)
  2276. MAPREDUCE-2224. Fix synchronization bugs in JvmManager. (todd)
  2277. MAPREDUCE-714. JobConf.findContainingJar unescapes unnecessarily on linux (todd)
  2278. MAPREDUCE-2096. Secure local filesystem IO from symlink vulnerabilities (todd)
  2279. MAPREDUCE-2234. If Localizer can't create task log directory, it should fail
  2280. on the spot. (todd)
  2281. MAPREDUCE-2219. JobTracker should not try to remove mapred.system.dir
  2282. during startup. (todd)
  2283. MAPREDUCE-2207. Task-cleanup task should not be scheduled on the node that
  2284. the task just failed. (Liyin Liang via schen)
  2285. MAPREDUCE-2084. Remove deprecate annotation for package file. The package
  2286. classes themselves are already deprecated. This removes an Eclipse error.
  2287. (tomwhite via nigel)
  2288. MAPREDUCE-2248. DistributedRaidFileSystem should unraid only the corrupt
  2289. block (Ramkumar Vadali via schen)
  2290. MAPREDUCE-1085. For tasks, "ulimit -v -1" is being run when user doesn't
  2291. specify a ulimit (todd)
  2292. MAPREDUCE-2282. Fix TestMRServerPorts for the changes in
  2293. TestHDFSServerPorts. (shv via szetszwo)
  2294. MAPREDUCE-2238. Fix permissions handling to avoid leaving undeletable
  2295. directories in local dirs. (todd)
  2296. MAPREDUCE-2277. TestCapacitySchedulerWithJobTracker needs to wait for jobs
  2297. to complete before testing status. (todd)
  2298. MAPREDUCE-2253. Servlets should specify content type (todd)
  2299. MAPREDUCE-2283. Add timeout for Raid Tests (Ramkumar Vadali via schen)
  2300. MAPREDUCE-1754. Replace mapred.persmissions.supergroup with an
  2301. acl : mapreduce.cluster.administrators (Amareshwari Sriramadasu via shv)
  2302. MAPREDUCE-2256. FairScheduler fairshare preemption from multiple pools may
  2303. preempt all tasks from one pool causing that pool to go below fairshare.
  2304. (Priyo Mustafi via shv)
  2305. MAPREDUCE-2281. MR part of HADOOP-6642. (Chris Douglas, Po Cheung via shv)
  2306. MAPREDUCE-2200. TestUmbilicalProtocolWithJobToken is failing without Krb
  2307. evironment: needs to be conditional. (cos)
  2308. MAPREDUCE-2077. Resolve name clash in the deprecated
  2309. o.a.h.util.MemoryCalculatorPlugin (Luke Lu via shv)
  2310. MAPREDUCE-2188. The new API MultithreadedMapper doesn't initialize
  2311. RecordReader. (Owen O'Malley via shv)
  2312. MAPREDUCE-1915. Fix IndexOutOfBoundsException in IndexCache.
  2313. (Priyo Mustafi via shv)
  2314. MAPREDUCE-1974. Fix multiple preemtions of the same task in FairScheduler.
  2315. (Scott Chen via shv)
  2316. MAPREDUCE-2304. Fix TestMRCLI to allow hostname with a hyphen (-).
  2317. (Priyo Mustafi via shv)
  2318. MAPREDUCE-1825. jobqueue_details.jsp and FairSchedulerServelet should not
  2319. call finishedMaps and finishedReduces when job is not initialized.
  2320. (Scott Chen via shv)
  2321. MAPREDUCE-2285. MiniMRCluster does not start after ant test-patch
  2322. (todd)
  2323. MAPREDUCE-2315. javadoc is failing in nightly build (todd)
  2324. MAPREDUCE-2054. Hierarchical queue implementation broke dynamic queue
  2325. addition in Dynamic Scheduler. (Thomas Sandholm via tomwhite)
  2326. MAPREDUCE-2272. Job ACL file should not be executable
  2327. (Harsh J Chouraria via todd)
  2328. MAPREDUCE-2241. ClusterWithLinuxTaskController should accept relative path
  2329. on the command line. (todd)
  2330. MAPREDUCE-2251. Remove unused mapreduce.job.userhistorylocation config.
  2331. (Harsh J Chouraria via todd)
  2332. MAPREDUCE-2284. TestLocalRunner.testMultiMaps times out (todd)
  2333. MAPREDUCE-2336. Tool-related packages should be in the Tool javadoc group.
  2334. (tomwhite)
  2335. MAPREDUCE-2394. JUnit output format doesn't propagate into raid contrib
  2336. build. (todd)
  2337. MAPREDUCE-2392. TaskTracker shutdown in the tests sometimes take 60s.
  2338. (tomwhite)
  2339. MAPREDUCE-2437. SLive uses only part* files to generating the final report.
  2340. (shv)
  2341. MAPREDUCE-2428. start-mapred.sh script fails if HADOOP_HOME is not set.
  2342. (tomwhite via eli)
  2343. MAPREDUCE-2445. Fix TestMiniMRWithDFSWithDistinctUsers to be a valid test.
  2344. (todd)
  2345. MAPREDUCE-2457. Job submission should inject group.name on the JobTracker
  2346. (Alejandro Abdelnur via todd)
  2347. MAPREDUCE-2472. Extra whitespace in mapred.child.java.opts breaks JVM
  2348. initialization. (Aaron T. Myers via todd)
  2349. MAPREDUCE-2222. Ivy resolve force mode should be turned off by default.
  2350. (Luke Lu via tomwhite)
  2351. MAPREDUCE-2486. Incorrect snapshot dependency published in .pom files
  2352. (todd)
  2353. MAPREDUCE-2327. MapTask doesn't need to put username information in
  2354. SpillRecord. (todd via tomwhite)
  2355. MAPREDUCE-2515. MapReduce code references some deprecated options
  2356. (Ari Rabkin via todd)
  2357. MAPREDUCE-2487. ChainReducer uses MAPPER_BY_VALUE instead of
  2358. REDUCER_BY_VALUE. (Devaraj K via todd)
  2359. MAPREDUCE-2185. Fix infinite loop at creating splits using
  2360. CombineFileInputFormat. (Ramkumar Vadali via schen)
  2361. MAPREDUCE-2571. CombineFileInputFormat.getSplits throws a
  2362. java.lang.ArrayStoreException. (Bochun Bai via todd)
  2363. MAPREDUCE-2767. Remove Linux task-controller. (Milind Bhandarkar via shv)
  2364. MAPREDUCE-2991. queueinfo.jsp fails to show queue status for Capacity
  2365. scheduler if queue names contain special symbols. (Priyo Mustafi via shv)
  2366. MAPREDUCE-2531. Fixed jobcontrol to downgrade JobID. (Robert Evans via
  2367. acmurthy)
  2368. MAPREDUCE-3139. SlivePartitioner generates negative partitions. (jghoman)
  2369. Release 0.21.1 - Unreleased
  2370. NEW FEATURES
  2371. MAPREDUCE-2040. Forrest Documentation for Dynamic Priority Scheduler.
  2372. (Thomas Sandholm via tomwhite)
  2373. BUG FIXES
  2374. MAPREDUCE-1897. trunk build broken on compile-mapred-test (cos)
  2375. MAPREDUCE-1280. Update Eclipse plugin to the new eclipse.jdt API.
  2376. (Alex Kozlov via szetszwo)
  2377. MAPREDUCE-1984. herriot TestCluster fails because exclusion is not there
  2378. (Balaji Rajagopalan via cos)
  2379. MAPREDUCE-2090. Clover build doesn't generate per-test coverage. (cos)
  2380. MAPREDUCE-2134. ant binary-system is broken in mapreduce project. (cos)
  2381. MAPREDUCE-1905. Fixes Context.setStatus() and progress() apis.
  2382. (amareshwari)
  2383. MAPREDUCE-1809. Ant build changes for Streaming system tests in contrib
  2384. projects. (Vinay Kumar Thota via amareshwari)
  2385. MAPREDUCE-2223. TestMRCLI might fail on Ubuntu with default /etc/hosts
  2386. (cos)
  2387. MAPREDUCE-2228. Remove java5 dependencies from build. (cos)
  2388. MAPREDUCE-1929. Allow artifacts to be published to the staging Apache Nexus
  2389. Maven Repository. (tomwhite)
  2390. MAPREDUCE-2317. Fix a NPE in HadoopArchives. (Devaraj K via szetszwo)
  2391. MAPREDUCE-2127. mapreduce trunk builds are filing on hudson.
  2392. (Bruno Mahé via eli)
  2393. MAPREDUCE-2779. JobSplitWriter.java can't handle large job.split file.
  2394. (Ming Ma via shv)
  2395. Release 0.21.0 - 2010-08-13
  2396. INCOMPATIBLE CHANGES
  2397. MAPREDUCE-516. Fix the starvation problem in the Capacity Scheduler
  2398. when running High RAM Jobs. (Arun Murthy via yhemanth)
  2399. MAPREDUCE-358. Change org.apache.hadoop.examples. AggregateWordCount
  2400. and org.apache.hadoop.examples.AggregateWordHistogram to use new
  2401. mapreduce api. (Amareshwari Sriramadasu via sharad)
  2402. MAPREDUCE-245. Change Job and jobcontrol classes to use the List interface
  2403. rather than ArrayList in APIs. (Tom White via cdouglas)
  2404. MAPREDUCE-766. Enhanced list-blacklisted-trackers to display reasons
  2405. for blacklisting a node. (Sreekanth Ramakrishnan via yhemanth)
  2406. MAPREDUCE-817. Add a cache for retired jobs with minimal job info and
  2407. provide a way to access history file url. (sharad)
  2408. MAPREDUCE-711. Moved Distributed Cache from Common to Map/Reduce
  2409. project. (Vinod Kumar Vavilapalli via yhemanth)
  2410. MAPREDUCE-895. Per the contract elucidated in HADOOP-6201, throw
  2411. FileNotFoundException from FileSystem::listStatus rather than returning
  2412. null. (Jakob Homan via cdouglas)
  2413. MAPREDUCE-479. Provide full task id to map output servlet rather than the
  2414. reduce id, only. (Jiaqi Tan via cdouglas)
  2415. MAPREDUCE-873. Simplify job recovery. Incomplete jobs are resubmitted on
  2416. jobtracker restart. Removes a public constructor in JobInProgress. (sharad)
  2417. HADOOP-6230. Moved process tree and memory calculator related classes from
  2418. Common to Map/Reduce. (Vinod Kumar Vavilapalli via yhemanth)
  2419. MAPREDUCE-157. Refactor job history APIs and change the history format to
  2420. JSON. (Jothi Padmanabhan via sharad)
  2421. MAPREDUCE-849. Rename configuration properties. (Amareshwari Sriramadasu
  2422. via sharad)
  2423. MAPREDUCE-1287. Only call the partitioner with more than one reducer.
  2424. (cdouglas)
  2425. MAPREDUCE-1385. Use the new UserGroupInformation from HADOOP-6299.
  2426. (ddas via omalley)
  2427. MAPREDUCE-1493. Authorization for job-history pages. (vinodkv)
  2428. MAPREDUCE-1607. Task controller may not set permissions for a
  2429. task cleanup attempt's log directory (Amareshwari Sriramadasu via vinodkv)
  2430. MAPREDUCE-1683. Remove JNI calls from ClusterStatus cstr. (Arun Murthy and
  2431. Luke Lu via cdouglas)
  2432. MAPREDUCE-1855. Makes the refresh methods (for groups and proxy users)
  2433. independent of the client side configuration. (Boris Shkolnik via ddas)
  2434. NEW FEATURES
  2435. MAPREDUCE-1774. Large-scale Automated Framework (Sharad Agarwal, Sreekanth
  2436. Ramakrishnan, Konstantin Boudnik, at all via cos)
  2437. MAPREDUCE-706. Support for FIFO pools in the fair scheduler.
  2438. (Matei Zaharia)
  2439. MAPREDUCE-546. Provide sample fair scheduler config file in conf/ and use
  2440. it by default if no other config file is specified. (Matei Zaharia)
  2441. MAPREDUCE-551. Preemption support in the Fair Scheduler. (Matei Zaharia)
  2442. MAPREDUCE-567. Add a new example MR that always fails. (Philip Zeyliger
  2443. via tomwhite)
  2444. MAPREDUCE-211. Provides ability to run a health check script on the
  2445. tasktracker nodes and blacklist nodes if they are unhealthy.
  2446. (Sreekanth Ramakrishnan via yhemanth)
  2447. MAPREDUCE-637. Add an example, distbbp, which able to compute the n th bit
  2448. of Pi for some large n. (szetszwo)
  2449. MAPREDUCE-532. Provide a way to limit the number of used slots
  2450. per queue in the capacity scheduler.
  2451. (Rahul Kumar Singh via yhemanth)
  2452. MAPREDUCE-467. Provide ability to collect statistics about total tasks
  2453. and succeeded tasks in different time windows. (sharad)
  2454. MAPREDUCE-740. Log a job-summary at the end of a job, while allowing it
  2455. to be configured to use a custom appender if desired. (acmurthy)
  2456. MAPREDUCE-814. Provide a way to configure completed job history files
  2457. to be on HDFS. (sharad)
  2458. MAPREDUCE-800. MRUnit should support the new API. (Aaron Kimball via
  2459. tomwhite)
  2460. MAPREDUCE-798. MRUnit should be able to test a succession of MapReduce
  2461. passes. (Aaron Kimball via tomwhite)
  2462. MAPREDUCE-768. Provide an option to dump jobtracker configuration in JSON
  2463. format to standard output. (V.V.Chaitanya Krishna via yhemanth)
  2464. MAPREDUCE-824. Add support for a hierarchy of queues in the capacity
  2465. scheduler. (Rahul Kumar Singh via yhemanth)
  2466. MAPREDUCE-751. Add Rumen, a tool for extracting statistics from job tracker
  2467. logs and generating job traces for simulation and analysis. (Dick King via
  2468. cdouglas)
  2469. MAPREDUCE-830. Add support for splittable compression to TextInputFormats.
  2470. (Abdul Qadeer via cdouglas)
  2471. MAPREDUCE-861. Add support for hierarchical queues in the Map/Reduce
  2472. framework. (Rahul Kumar Singh via yhemanth)
  2473. MAPREDUCE-776. Add Gridmix, a benchmark processing Rumen traces to simulate
  2474. a measured mix of jobs on a cluster. (cdouglas)
  2475. MAPREDUCE-862. Enhance JobTracker UI to display hierarchical queues.
  2476. (V.V.Chaitanya Krishna via yhemanth)
  2477. MAPREDUCE-777. Brand new apis to track and query jobs as a
  2478. replacement for JobClient. (Amareshwari Sriramadasu via acmurthy)
  2479. MAPREDUCE-775. Add native and streaming support for Vertica as an input
  2480. or output format taking advantage of parallel read and write properties of
  2481. the DBMS. (Omer Trajman via ddas)
  2482. MAPREDUCE-679. XML-based metrics as JSP servlet for JobTracker.
  2483. (Aaron Kimball via tomwhite)
  2484. MAPREDUCE-980. Modify JobHistory to use Avro for serialization. (cutting)
  2485. MAPREDUCE-728. Add Mumak, a Hadoop map/reduce simulator. (Arun C Murthy,
  2486. Tamas Sarlos, Anirban Dasgupta, Guanying Wang, and Hong Tang via cdouglas)
  2487. MAPREDUCE-1383. Automates fetching of delegation tokens in File*Formats
  2488. Distributed Cache and Distcp. Also, provides a config
  2489. mapreduce.job.hdfs-servers that the jobs can populate with a comma
  2490. separated list of namenodes. The job client automatically fetches
  2491. delegation tokens from those namenodes. (Boris Shkolnik via ddas)
  2492. MAPREDUCE-698. Per-pool task limits for the fair scheduler.
  2493. (Kevin Peterson via matei)
  2494. MAPREDUCE-1026. Does mutual authentication of the shuffle
  2495. transfers using a shared JobTracker generated key.
  2496. (Boris Shkolnik via ddas)
  2497. MAPREDUCE-744. Introduces the notion of a public distributed cache.
  2498. (Devaraj Das)
  2499. MAPREDUCE-1338. Introduces the notion of token cache using which
  2500. tokens and secrets can be sent by the Job client to the JobTracker.
  2501. (Boris Shkolnik via ddas)
  2502. HDFS-503. This patch implements an optional layer over HDFS that
  2503. implements offline erasure-coding. It can be used to reduce the
  2504. total storage requirements of HDFS. (dhruba)
  2505. MAPREDUCE-1432. Adds hooks in the jobtracker and tasktracker
  2506. for loading the tokens in the user's ugi. This is required
  2507. for the copying of files from the hdfs. (ddas)
  2508. MAPREDUCE-1335. Adds SASL Kerberos/Digest authentication in MapReduce.
  2509. (Kan Zhang via ddas)
  2510. MAPREDUCE-1464. Makes a compatible change in JobTokenIdentifier to
  2511. account for HADOOP-6510. (Jitendra Nath Pandey via ddas)
  2512. MAPREDUCE-1433. Add a delegation token for MapReduce. (omalley)
  2513. MAPREDUCE-1307. Introduces the Job level ACLs feature.
  2514. (Vinod Kumar Vavilapalli via ddas)
  2515. MAPREDUCE-1430. JobTracker automatically renews delegation tokens for jobs.
  2516. (Boris Shkolnik via ddas)
  2517. MAPREDUCE-1455. Introduces job-level authorization for mapreduce servlets.
  2518. (Ravi Gummadi via vinodkv)
  2519. IMPROVEMENTS
  2520. MAPREDUCE-463. Makes job setup and cleanup tasks as optional.
  2521. (Amareshwari Sriramadasu via sharad)
  2522. MAPREDUCE-502. Allow jobtracker to be configured with zero completed jobs
  2523. in memory. (Amar Kamat via sharad)
  2524. MAPREDUCE-416. Moves the history file to a "done" folder whenever a job
  2525. completes. (Amar Kamat via ddas)
  2526. MAPREDUCE-646. Increase srcfilelist replication number in dictcp job.
  2527. (Ravi Gummadi via szetszwo)
  2528. HADOOP-6106. Updated hadoop-core and test jars from hudson trunk
  2529. build #12. (Giridharan Kesavan)
  2530. MAPREDUCE-642. A option to distcp that allows preserving the full
  2531. source path of a file in the specified destination directory.
  2532. (Rodrigo Schmidt via dhruba)
  2533. MAPREDUCE-686. Move TestSpeculativeExecution.Fake* into a separate class
  2534. so that it can be used by other tests. (Jothi Padmanabhan via sharad)
  2535. MAPREDUCE-625. Modify TestTaskLimits to improve execution time.
  2536. (Jothi Padmanabhan via sharad)
  2537. MAPREDUCE-465. Deprecate o.a.h.mapred.lib.MultithreadedMapRunner and add
  2538. test for o.a.h.mapreduce.lib.MultithreadedMapper.
  2539. (Amareshwari Sriramadasu via sharad)
  2540. MAPREDUCE-701. Improves the runtime of the TestRackAwareTaskPlacement
  2541. by making it a unit test. (Jothi Padmanabhan via ddas)
  2542. MAPREDUCE-371. Change KeyFieldBasedComparator and KeyFieldBasedPartitioner
  2543. to use new api. (Amareshwari Sriramadasu via sharad)
  2544. MAPREDUCE-623. Resolve javac warnings in mapreduce. (Jothi Padmanabhan
  2545. via sharad)
  2546. MAPREDUCE-655. Change KeyValueLineRecordReader and KeyValueTextInputFormat
  2547. to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
  2548. MAPREDUCE-632. Merge TestCustomOutputCommitter with
  2549. TestCommandLineJobSubmission. (Jothi Padmanabhan via sharad)
  2550. MAPREDUCE-627. Improves execution time of TestTrackerBlacklistAcrossJobs.
  2551. (Jothi Padmanabhan via ddas)
  2552. MAPREDUCE-630. Improves execution time of TestKillCompletedJob.
  2553. (Jothi Padmanabhan via ddas)
  2554. MAPREDUCE-626. Improves the execution time of TestLostTracker.
  2555. (Jothi Padmanabhan via ddas)
  2556. MAPREDUCE-353. Makes the shuffle read and connection timeouts
  2557. configurable. (Ravi Gummadi via ddas)
  2558. MAPREDUCE-739. Allow relative paths to be created in archives. (Mahadev
  2559. Konar via cdouglas)
  2560. MAPREDUCE-772. Merge HADOOP-4010 changes to LineRecordReader into mapreduce
  2561. package. (Abdul Qadeer via cdouglas)
  2562. MAPREDUCE-785. Separate sub-test of TestReduceFetch to be included in
  2563. MR-670. (Jothi Padmanabhan via cdouglas)
  2564. MAPREDUCE-784. Modify TestUserDefinedCounters to use LocalJobRunner
  2565. instead of MiniMR. (Jothi Padmanabhan via sharad)
  2566. HADOOP-6160. Fix releaseaudit target to run on specific directories.
  2567. (gkesavan)
  2568. MAPREDUCE-782. Use PureJavaCrc32 in SpillRecord. (Todd Lipcon via
  2569. szetszwo)
  2570. MAPREDUCE-369. Change org.apache.hadoop.mapred.lib.MultipleInputs to
  2571. use new api. (Amareshwari Sriramadasu via sharad)
  2572. MAPREDUCE-373. Change org.apache.hadoop.mapred.lib.FieldSelectionMapReduce
  2573. to use new api. (Amareshwari Sriramadasu via sharad)
  2574. MAPREDUCE-628. Improves the execution time of TestJobInProgress.
  2575. (Jothi Padmanabhan via ddas)
  2576. MAPREDUCE-793. Creates a new test that consolidates a few tests to
  2577. include in the commit-test list. (Jothi Padmanabhan via ddas)
  2578. MAPREDUCE-797. Adds combiner support to MRUnit MapReduceDriver.
  2579. (Aaron Kimball via johan)
  2580. MAPREDUCE-656. Change org.apache.hadoop.mapred.SequenceFile* classes
  2581. to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
  2582. MAPREDUCE-670. Creates ant target for 10 mins patch test build.
  2583. (Jothi Padmanabhan via gkesavan)
  2584. MAPREDUCE-375. Change org.apache.hadoop.mapred.lib.NLineInputFormat
  2585. and org.apache.hadoop.mapred.MapFileOutputFormat to use new api.
  2586. (Amareshwari Sriramadasu via ddas)
  2587. MAPREDUCE-779. Added node health failure counts into
  2588. JobTrackerStatistics. (Sreekanth Ramakrishnan via yhemanth)
  2589. MAPREDUCE-842. Setup secure permissions for localized job files,
  2590. intermediate outputs and log files on tasktrackers.
  2591. (Vinod Kumar Vavilapalli via yhemanth)
  2592. MAPREDUCE-478. Allow map and reduce jvm parameters, environment variables
  2593. and ulimit to be set separately.
  2594. Configuration changes:
  2595. add mapred.map.child.java.opts
  2596. add mapred.reduce.child.java.opts
  2597. add mapred.map.child.env
  2598. add mapred.reduce.child.ulimit
  2599. add mapred.map.child.env
  2600. add mapred.reduce.child.ulimit
  2601. deprecated mapred.child.java.opts
  2602. deprecated mapred.child.env
  2603. deprecated mapred.child.ulimit
  2604. (acmurthy)
  2605. MAPREDUCE-767. Remove the dependence on the CLI 2.0 snapshot.
  2606. (Amar Kamat via omalley)
  2607. MAPREDUCE-712. Minor efficiency tweaks to RandomTextWriter. (cdouglas)
  2608. MAPREDUCE-870. Remove the job retire thread and the associated
  2609. config parameters. (sharad)
  2610. MAPREDUCE-874. Rename the PiEstimator example to QuasiMonteCarlo.
  2611. (szetszwo)
  2612. MAPREDUCE-336. Allow logging level of map/reduce tasks to be configurable.
  2613. Configuration changes:
  2614. add mapred.map.child.log.level
  2615. add mapred.reduce.child.log.level
  2616. (acmurthy)
  2617. MAPREDUCE-355. Update mapred.join package to use the new API. (Amareshwari
  2618. Sriramadasu via cdouglas)
  2619. HADOOP-6184. Updated hadoop common and test jars to get the new API
  2620. in Configuration for dumping in JSON format from Hudson trunk build #68.
  2621. (yhemanth)
  2622. MAPREDUCE-476. Extend DistributedCache to work locally (LocalJobRunner).
  2623. (Philip Zeyliger via tomwhite)
  2624. MAPREDUCE-825. JobClient completion poll interval of 5s causes slow tests
  2625. in local mode. (Aaron Kimball via tomwhite)
  2626. MAPREDUCE-910. Support counters in MRUnit. (Aaron Kimball via cdouglas)
  2627. MAPREDUCE-788. Update gridmix2 to use the new API (Amareshwari Sriramadasu
  2628. via cdouglas)
  2629. MAPREDUCE-875. Make DBRecordReader execute queries lazily. (Aaron Kimball
  2630. via enis)
  2631. MAPREDUCE-318. Modularizes the shuffle code. (Jothi Padmanabhan and
  2632. Arun Murthy via ddas)
  2633. MAPREDUCE-936. Allow a load difference for fairshare scheduler.
  2634. (Zheng Shao via dhruba)
  2635. MAPREDUCE-370. Update MultipleOutputs to use the API, merge funcitonality
  2636. of MultipleOutputFormat. (Amareshwari Sriramadasu via cdouglas)
  2637. MAPREDUCE-898. Changes DistributedCache to use the new API.
  2638. (Amareshwari Sriramadasu via ddas)
  2639. MAPREDUCE-144. Includes dump of the process tree in task diagnostics when
  2640. a task is killed due to exceeding memory limits.
  2641. (Vinod Kumar Vavilapalli via yhemanth)
  2642. MAPREDUCE-945. Modifies MRBench and TestMapRed to use ToolRunner so that
  2643. options such as queue name can be passed via command line.
  2644. (Sreekanth Ramakrishnan via yhemanth)
  2645. MAPREDUCE-963. Deprecate o.a.h.mapred.FileAlreadyExistsException and
  2646. replace it with o.a.h.fs.FileAlreadyExistsException. (Boris Shkolnik
  2647. via szetszwo)
  2648. MAPREDUCE-960. Remove an unnecessary intermediate copy and obsolete API
  2649. from KeyValueLineRecordReader. (cdouglas)
  2650. MAPREDUCE-930. Modify Rumen to resolve paths in the canonical way, rather
  2651. than defaulting to the local filesystem. (cdouglas)
  2652. MAPREDUCE-944. Extend the LoadManager API of the fair-share scheduler
  2653. to support regulating tasks for a job based on resources currently in use
  2654. by that job. (dhruba)
  2655. MAPREDUCE-973. Move FailJob and SleepJob from examples to test. (cdouglas
  2656. via omalley)
  2657. MAPREDUCE-966. Modify Rumen to clean up interfaces and simplify integration
  2658. with other tools. (Hong Tang via cdouglas)
  2659. MAPREDUCE-856. Setup secure permissions for distributed cache files.
  2660. (Vinod Kumar Vavilapalli via yhemanth)
  2661. MAPREDUCE-885. More efficient SQL queries for DBInputFormat. (Aaron Kimball
  2662. via enis)
  2663. MAPREDUCE-284. Enables ipc.client.tcpnodelay in Tasktracker's Child.
  2664. (Ravi Gummadi via sharad)
  2665. MAPREDUCE-916. Split the documentation to match the project split.
  2666. (Corinne Chandel via omalley)
  2667. MAPREDUCE-649. Validate a copy by comparing the source and destination
  2668. checksums in distcp. Also adds an intra-task retry mechanism for errors
  2669. detected during the copy. (Ravi Gummadi via cdouglas)
  2670. MAPREDUCE-654. Add a -dryrun option to distcp printing a summary of the
  2671. file data to be copied, without actually performing the copy. (Ravi Gummadi
  2672. via cdouglas)
  2673. MAPREDUCE-664. Display the number of files deleted by distcp when the
  2674. -delete option is specified. (Ravi Gummadi via cdouglas)
  2675. MAPREDUCE-781. Let the name of distcp jobs be configurable. (Venkatesh S
  2676. via cdouglas)
  2677. MAPREDUCE-975. Add an API in job client to get the history file url for
  2678. a given job id. (sharad)
  2679. MAPREDUCE-905. Add Eclipse launch tasks for MapReduce. (Philip Zeyliger
  2680. via tomwhite)
  2681. MAPREDUCE-277. Makes job history counters available on the job history
  2682. viewers. (Jothi Padmanabhan via ddas)
  2683. MAPREDUCE-893. Provides an ability to refresh queue configuration
  2684. without restarting the JobTracker.
  2685. (Vinod Kumar Vavilapalli and Rahul Kumar Singh via yhemanth)
  2686. MAPREDUCE-1011. Add build.properties to svn and git ignore. (omalley)
  2687. MAPREDUCE-954. Change Map-Reduce context objects to be interfaces.
  2688. (acmurthy)
  2689. MAPREDUCE-639. Change Terasort example to reflect the 2009 updates.
  2690. (omalley)
  2691. MAPREDUCE-1063. Document gridmix benchmark. (cdouglas)
  2692. MAPREDUCE-931. Use built-in interpolation classes for making up task
  2693. runtimes in Rumen. (Dick King via cdouglas)
  2694. MAPREDUCE-1012. Mark Context interfaces as public evolving. (Tom White via
  2695. cdouglas)
  2696. MAPREDUCE-971. Document use of distcp when copying to s3, managing timeouts
  2697. in particular. (Aaron Kimball via cdouglas)
  2698. HDFS-663. DFSIO for append. (shv)
  2699. HDFS-641. Move all of the components that depend on map/reduce to
  2700. map/reduce. (omalley)
  2701. HADOOP-5107. Use Maven ant tasks to publish artifacts. (Giridharan Kesavan
  2702. via omalley)
  2703. MAPREDUCE-1229. Allow customization of job submission policy in Mumak.
  2704. (Hong Tang via cdouglas)
  2705. MAPREDUCE-1317. Reduce the memory footprint of Rumen objects by interning
  2706. host Strings. (Hong Tang via cdouglas)
  2707. MAPREDUCE-1097. Add support for Vertica 3.5 to its contrib module. (Omer
  2708. Trajman via cdouglas)
  2709. MAPREDUCE-1627. HadoopArchives should not uses a method in DistCp.
  2710. (szetszwo)
  2711. MAPREDUCE-1198. Alternatively schedule different types of tasks in
  2712. fair share scheduler. (Scott Chen via matei)
  2713. MAPREDUCE-707. Provide a jobconf property for explicitly assigning a job to
  2714. a pool in the Fair Scheduler. (Alan Heirich via matei)
  2715. MAPREDUCE-947. Added commitJob and abortJob apis to OutputCommitter.
  2716. Enhanced FileOutputCommitter to create a _SUCCESS file for successful
  2717. jobs. (Amar Kamat & Jothi Padmanabhan via acmurthy)
  2718. MAPREDUCE-1103. Added more metrics to Jobtracker. (sharad)
  2719. MAPREDUCE-1048. Add occupied/reserved slot usage summary on jobtracker UI.
  2720. (Amareshwari Sriramadasu and Hemanth Yamijala via sharad)
  2721. MAPREDUCE-1090. Modified log statement in TaskMemoryManagerThread to
  2722. include task attempt id. (yhemanth)
  2723. MAPREDUCE-1189. Reduce ivy console output to ovservable level (cos)
  2724. MAPREDUCE-1167. ProcfsBasedProcessTree collects rss memory information.
  2725. (Scott Chen via dhruba)
  2726. MAPREDUCE-1231. Added a new DistCp option, -skipcrccheck, so that the CRC
  2727. check during setup can be skipped. (Jothi Padmanabhan via szetszwo)
  2728. MAPREDUCE-1190. Add package documentation for BBP example.
  2729. (Tsz Wo (Nicholas) Sze via cdouglas)
  2730. MAPREDUCE-1119. When tasks fail to report status, show tasks's stack dump
  2731. before killing. (Aaron Kimball via tomwhite)
  2732. MAPREDUCE-1185. Redirect running job url to history url if job is already
  2733. retired. (Amareshwari Sriramadasu and Sharad Agarwal via sharad)
  2734. MAPREDUCE-1050. Introduce a mock object testing framework. (tomwhite)
  2735. MAPREDUCE-1084. Implementing aspects development and fault injeciton
  2736. framework for MapReduce. (Sreekanth Ramakrishnan via cos)
  2737. MAPREDUCE-1209. Move common specific part of the test TestReflectionUtils
  2738. out of mapred into common. (Todd Lipcon via tomwhite)
  2739. MAPREDUCE-967. TaskTracker does not need to fully unjar job jars.
  2740. (Todd Lipcon via tomwhite)
  2741. MAPREDUCE-1083. Changes in MapReduce so that group information of users
  2742. can be refreshed in the JobTracker via command line.
  2743. (Boris Shkolnik via ddas)
  2744. MAPREDUCE-181. Changes the job submission process to be secure.
  2745. (Devaraj Das)
  2746. MAPREDUCE-1250. Refactors the JobToken to use Common's Token interface.
  2747. (Kan Zhang via ddas)
  2748. MAPREDUCE-896. Enhance tasktracker to cleanup files that might have
  2749. been created by user tasks with non-writable permissions.
  2750. (Ravi Gummadi via yhemanth)
  2751. MAPREDUCE-372. Change org.apache.hadoop.mapred.lib.ChainMapper/Reducer
  2752. to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
  2753. MAPREDUCE-1295. Add a tool in Rumen for folding and manipulating job
  2754. traces. (Dick King via cdouglas)
  2755. MAPREDUCE-1302. TrackerDistributedCacheManager deletes file
  2756. asynchronously, thus reducing task initialization delays.
  2757. (Zheng Shao via dhruba)
  2758. MAPREDUCE-1218. TaskTrackers send cpu and memory usage of
  2759. node to JobTracker. (Scott Chen via dhruba)
  2760. MAPREDUCE-847. Fix Releaseaudit warning count to zero
  2761. (Giridharan Kesavan)
  2762. MAPREDUCE-1337. Use generics in StreamJob to improve readability of that
  2763. class. (Kay Kay via cdouglas)
  2764. MAPREDUCE-361. Port terasort example to the new mapreduce API. (Amareshwari
  2765. Sriramadasu via cdouglas)
  2766. MAPREDUCE-1367. LocalJobRunner should support parallel mapper execution.
  2767. (Aaron Kimball via tomwhite)
  2768. MAPREDUCE-64. Eliminate io.sort.record.percent from MapTask configuration.
  2769. (cdouglas)
  2770. MAPREDUCE-1440. Replace the long user name in MapReduce with the local
  2771. name. (omalley)
  2772. MAPREDUCE-1470. Move delegation tokens from HDFS to Common so that
  2773. MapReduce can use them too. (omalley)
  2774. MAPREDUCE-1425. Reduce memory usage by archive. (mahadev via szetszwo)
  2775. MAPREDUCE-1441. Trim whitespace from directory lists pulled from the
  2776. configuration. (Todd Lipcon via cdouglas)
  2777. MAPREDUCE-1309. Refactor Rumen trace generator to improve code structure
  2778. and add extensible support for log formats. (Dick King via cdouglas)
  2779. MAPREDUCE-1503. Delegation token renewing and cancelling should provide
  2780. meaningful exceptions when there are failures instead of returning
  2781. false. (omalley)
  2782. HADOOP-6579. Upgrade commons-codec library to 1.4. (omalley)
  2783. MAPREDUCE-1423. Improve performance of CombineFileInputFormat when multiple
  2784. pools are configured. (Dhruba Borthakur via zshao)
  2785. MAPREDUCE-1454. Quote user supplied strings in Tracker servlets. (cdouglas)
  2786. MAPREDUCE-1408. Add customizable job submission policies to Gridmix. (Rahul
  2787. Singh via cdouglas)
  2788. MAPREDUCE-1527. Better warning logged when mapred.queue.names is
  2789. overshadowed by mapred-queues.xml. (Hong Tang via acmurthy)
  2790. MAPREDUCE-1403. Save the size and number of distributed cache artifacts in
  2791. the configuration. (Arun Murthy via cdouglas)
  2792. MAPREDUCE-1482. Truncate state string and diagnostic information in
  2793. TaskStatus. (Amar Kamat via szetszwo)
  2794. MAPREDUCE-1593. [Rumen] Improvements to random seed generation (tamas via
  2795. mahadev)
  2796. MAPREDUCE-1460. Oracle support in DataDrivenDBInputFormat.
  2797. (Aaron Kimball via tomwhite)
  2798. MAPREDUCE-1569. Pass configuration through mocked contexts in MRUnit.
  2799. (Chris White via cdouglas)
  2800. MAPREDUCE-1590. Move HarFileSystem from Hadoop Common to Mapreduce tools.
  2801. (mahadev)
  2802. MAPREDUCE-1629. Get rid of fakeBlockLocations() on HarFileSystem, since
  2803. it's not used (mahadev)
  2804. MAPREDUCE-1489. DataDrivenDBInputFormat should not query the database
  2805. when generating only one split. (Aaron Kimball via tomwhite)
  2806. MAPREDUCE-1514. Add documentation on replication, permissions, new options,
  2807. limitations and internals of har. (mahadev via szetszwo)
  2808. MAPREDUCE-1428. Make block size and the size of archive created files
  2809. configurable. (mahadev via szetszwo)
  2810. MAPREDUCE-1656. JobStory should provide queue info. (hong via mahadev)
  2811. MAPREDUCE-1466. Record number of files processed in FileInputFormat in the
  2812. Configuration for offline analysis. (Luke Lu and Arun Murthy via cdouglas)
  2813. MAPREDUCE-1538. TrackerDistributedCacheManager manages the
  2814. number of files. (Scott Chen via dhruba)
  2815. MAPREDUCE-1673. Scripts to start and stop RaidNode.
  2816. (Rodrigo Schmidt via dhruba)
  2817. MAPREDUCE-1659. RaidNode writes temp files on configured tmp directory and
  2818. add random numbers to their names to avoid conflicts
  2819. (Rodrigo Schmidt via dhruba)
  2820. MAPREDUCE-1221. Allow admins to control physical memory limits per-task
  2821. and per-node. (Scott Chen via acmurthy)
  2822. MAPREDUCE-1065. Update mapred tutorial to use the new API. (Aaron Kimball
  2823. via cdouglas)
  2824. MAPREDUCE-1304. Add a task counter tracking time spent in GC. (Aaron
  2825. Kimball via cdouglas)
  2826. MAPREDUCE-1570. Add grouping comparators to MRUnit. (Chris White via
  2827. cdouglas)
  2828. MAPREDUCE-1650. Exclude Private elements from generated MapReduce
  2829. Javadoc. (tomwhite)
  2830. MAPREDUCE-1625. Improve grouping of packages in Javadoc. (tomwhite)
  2831. MAPREDUCE-1417. Forrest documentation should be updated to reflect
  2832. the changes in MAPREDUCE-744. (Ravi Gummadi via vinodkv)
  2833. MAPREDUCE-1568. TrackerDistributedCacheManager should clean up cache
  2834. in a background thread. (Scott Chen via zshao)
  2835. MAPREDUCE-1749. Move configuration strings out of JobContext so that it
  2836. can be made public stable. (omalley)
  2837. MAPREDUCE-1623. Apply audience and stability notations to Hadoop
  2838. Map-Reduce. (tomwhite via acmurthy)
  2839. MAPREDUCE-1751. Change MapReduce to depend on Hadoop 'common' artifacts
  2840. instead of 'core'. (tomwhite)
  2841. MAPREDUCE-1535. Replace usage of FileStatus#isDir(). (Eli Collins via
  2842. tomwhite)
  2843. MAPREDUCE-1832. Allow file sizes less than 1MB in DFSIO benchmark. (shv)
  2844. MAPREDUCE-1404. Move Cluster-Setup and Single-Node-Setup Docs from
  2845. MapReduce to Common. (tomwhite)
  2846. MAPREDUCE-1697. Document the behavior of -file option and deprecate it
  2847. in favour of -files option in streaming. (Amareshwari Sriramadasu
  2848. via vinodkv)
  2849. MAPREDUCE-1033. Resolve location of scripts and configuration files after
  2850. project split. (tomwhite)
  2851. MAPREDUCE-1018. Document changes to the memory management and scheduling
  2852. model. (Hemanth Yamijala via vinodkv)
  2853. MAPREDUCE-1896. [Herriot] New property for multi user list. (Vinay Thota
  2854. via cos)
  2855. MAPREDUCE-1812. New properties for suspend and resume process. (Vinay
  2856. Thota via cos)
  2857. OPTIMIZATIONS
  2858. MAPREDUCE-270. Fix the tasktracker to optionally send an out-of-band
  2859. heartbeat on task-completion for better job-latency. (acmurthy)
  2860. Configuration changes:
  2861. add mapreduce.tasktracker.outofband.heartbeat
  2862. MAPREDUCE-1186. Modified code in distributed cache to set permissions
  2863. only on required set of localized paths.
  2864. (Amareshwari Sriramadasu via yhemanth)
  2865. MAPREDUCE-1501. FileInputFormat supports multi-level, recursive
  2866. directory listing. (Zheng Shao via dhruba)
  2867. MAPREDUCE-1556. upgrade to Avro 1.3.0. (cutting via tomwhite)
  2868. MAPREDUCE-1613. Install/deploy source jars to Maven repo
  2869. (Patrick Angeles via ddas)
  2870. MAPREDUCE-1610. Forrest documentation should be updated to reflect
  2871. the changes in MAPREDUCE-856. (Ravi Gummadi via vinodkv)
  2872. MAPREDUCE-1853. Adds caching for TaskAttemptContext in MultipleOutputs.
  2873. (Torsten Curdt via amareshwari)
  2874. BUG FIXES
  2875. MAPREDUCE-878. Rename fair scheduler design doc to
  2876. fair-scheduler-design-doc.tex and add Apache license header (matei)
  2877. HADOOP-4687. MapReduce is split from Hadoop Core. It is a subproject under
  2878. Hadoop (Owen O'Malley)
  2879. HADOOP-6096. Fix Eclipse project and classpath files following project
  2880. split. (tomwhite)
  2881. MAPREDUCE-419. Reconcile mapred.userlog.limit.kb defaults in configuration
  2882. and code. (Philip Zeyliger via cdouglas)
  2883. MAPREDUCE-2. Fixes a bug in KeyFieldBasedPartitioner in handling empty
  2884. keys. (Amar Kamat via sharad)
  2885. MAPREDUCE-130. Delete the jobconf copy from the log directory of the
  2886. JobTracker when the job is retired. (Amar Kamat via sharad)
  2887. MAPREDUCE-657. Fix hardcoded filesystem problem in CompletedJobStatusStore.
  2888. (Amar Kamat via sharad)
  2889. MAPREDUCE-179. Update progress in new RecordReaders. (cdouglas)
  2890. MAPREDUCE-658. Replace NPE in distcp with a meaningful error message when
  2891. the source path does not exist. (Ravi Gummadi via cdouglas)
  2892. MAPREDUCE-671. Update ignore list to include untracked, generated
  2893. build artifacts and config files. (cdouglas)
  2894. MAPREDUCE-433. Use more reliable counters in TestReduceFetch. (cdouglas)
  2895. MAPREDUCE-124. Fix a bug in failure handling of abort task of
  2896. OutputCommiter. (Amareshwari Sriramadasu via sharad)
  2897. MAPREDUCE-694. Fix to add jsp-api jars to capacity-scheduler classpath.
  2898. (Giridharan Kesavan)
  2899. MAPREDUCE-702. Fix eclipse-plugin jar target (Giridharan Kesavan)
  2900. MAPREDUCE-522. Replace TestQueueCapacities with simpler test case to
  2901. test integration between capacity scheduler and MR framework.
  2902. (Sreekanth Ramakrishnan via yhemanth)
  2903. MAPREDUCE-683. Fixes an initialization problem in the JobHistory.
  2904. The initialization of JobHistoryFilesManager is now done in the
  2905. JobHistory.init call. (Amar Kamat via ddas)
  2906. MAPREDUCE-708. Fixes a bug to allow updating the reason for
  2907. blacklisting a node on the JobTracker UI.
  2908. (Sreekanth Ramakrishnan via yhemanth)
  2909. MAPREDUCE-709. Fixes message displayed for a blacklisted node where
  2910. the reason for blacklisting is due to the health check script
  2911. timing out. (Sreekanth Ramakrishnan via yhemanth)
  2912. MAPREDUCE-676. Existing diagnostic rules fail for MAP ONLY jobs.
  2913. (Suhas Gogate via tomwhite)
  2914. MAPREDUCE-722. Fixes a bug with tasktracker reservations for
  2915. high memory jobs in capacity scheduler.
  2916. (Vinod Kumar Vavilapalli via yhemanth)
  2917. HADOOP-6090. Updates gridmix script to use new mapreduce api output
  2918. format. (Amareshwari Sriramadasu via sharad)
  2919. MAPREDUCE-732. Removed spurious log statements in the node
  2920. blacklisting logic. (Sreekanth Ramakrishnan via yhemanth)
  2921. MAPREDUCE-734. Fix a ConcurrentModificationException in unreserving
  2922. unused reservations for a job when it completes.
  2923. (Arun Murthy and Sreekanth Ramakrishnan via yhemanth)
  2924. MAPREDUCE-733. Fix a RuntimeException while unreserving trackers
  2925. that are blacklisted for a job.
  2926. (Arun Murthy and Sreekanth Ramakrishnan via yhemanth)
  2927. MAPREDUCE-677. Fix timeout in TestNodeRefresh. (Amar Kamat via
  2928. sharad)
  2929. MAPREDUCE-153. Fix timeout in TestJobInProgressListener. (Amar
  2930. Kamat via sharad)
  2931. MAPREDUCE-742. Fix output messages and java comments in the Pi related
  2932. examples. (szetszwo)
  2933. MAPREDUCE-565. Fix partitioner to work with new API. (Owen O'Malley via
  2934. cdouglas)
  2935. MAPREDUCE-680. Fix so MRUnit can handle reuse of Writable objects.
  2936. (Aaron Kimball via johan)
  2937. MAPREDUCE-18. Puts some checks for cross checking whether a reduce
  2938. task gets the correct shuffle data. (Ravi Gummadi via ddas)
  2939. MAPREDUCE-771. Fix scheduling of setup and cleanup tasks to use
  2940. free slots instead of tasks for scheduling. (yhemanth)
  2941. MAPREDUCE-717. Fixes some corner case issues in speculative
  2942. execution heuristics. (Devaraj Das)
  2943. MAPREDUCE-716. Make DBInputFormat work with Oracle. (Aaron Kimball
  2944. via tomwhite)
  2945. MAPREDUCE-735. Fixes a problem in the KeyFieldHelper to do with
  2946. the end index for some inputs (Amar Kamat via ddas)
  2947. MAPREDUCE-682. Removes reservations on tasktrackers which are
  2948. blacklisted. (Sreekanth Ramakrishnan via yhemanth)
  2949. MAPREDUCE-743. Fixes a problem to do with progress reporting
  2950. in the map phase. (Ravi Gummadi via ddas)
  2951. MAPREDUCE-765. Eliminate the deprecated warnings introduced by H-5438.
  2952. (He Yongqiang via szetszwo)
  2953. MAPREDUCE-383. Fix a bug in Pipes combiner due to bytes count not
  2954. getting reset after the spill. (Christian Kunz via sharad)
  2955. MAPREDUCE-809. Fix job-summary logs to correctly record status of FAILED
  2956. and KILLED jobs. (acmurthy)
  2957. MAPREDUCE-792. Fix unchecked warnings in DBInputFormat. (Aaron Kimball
  2958. via szetszwo)
  2959. MAPREDUCE-760. Fix a timing issue in TestNodeRefresh. (Amar Kamat via
  2960. sharad)
  2961. MAPREDUCE-40. Keep memory management backwards compatible for job
  2962. configuration parameters and limits. (Rahul Kumar Singh via yhemanth)
  2963. MAPREDUCE-587. Fixes a OOM issue in TestStreamingExitStatus.
  2964. (Amar Kamat via ddas)
  2965. MAPREDUCE-408. Fixes an assertion problem in TestKillSubProcesses
  2966. (Ravi Gummadi via ddas)
  2967. MAPREDUCE-659. Fix gridmix2 compilation. (Giridharan Kesavan)
  2968. MAPREDUCE-796. Fixes a ClassCastException in an exception log in
  2969. MultiThreadedMapRunner. (Amar Kamat via ddas)
  2970. MAPREDUCE-808. Fixes a serialization problem in TypedBytes.
  2971. (Klaas Bosteels via ddas)
  2972. MAPREDUCE-845. Fix a findbugs heap size problem in build.xml and add
  2973. a new property findbugs.heap.size. (Lee Tucker via szetszwo)
  2974. MAPREDUCE-838. Fixes a problem in the way commit of task outputs
  2975. happens. The bug was that even if commit failed, the task would
  2976. be declared as successful. (Amareshwari Sriramadasu via ddas)
  2977. MAPREDUCE-813. Updates Streaming and M/R tutorial documents.
  2978. (Corinne Chandel via ddas)
  2979. MAPREDUCE-805. Fixes some deadlocks in the JobTracker due to the fact
  2980. the JobTracker lock hierarchy wasn't maintained in some JobInProgress
  2981. method calls. (Amar Kamat via ddas)
  2982. MAPREDUCE-799. Fixes so all of the MRUnit self-tests run.
  2983. (Aaron Kimball via johan)
  2984. MAPREDUCE-848. Fixes a problem to do with TestCapacityScheduler
  2985. failing (Amar Kamat via ddas)
  2986. MAPREDUCE-840. DBInputFormat leaves open transaction.
  2987. (Aaron Kimball via tomwhite)
  2988. MAPREDUCE-859. Adds Avro and its dependencies required by Hadoop
  2989. common. (Ravi Gummadi via sharad)
  2990. MAPREDUCE-867. Fix ivy conf to look for avro jar from maven repo.
  2991. (Giridharan Kesavan)
  2992. MAPREDUCE-877. Added avro as a dependency to contrib ivy settings.
  2993. (Tsz Wo (Nicholas) Sze via yhemanth)
  2994. MAPREDUCE-852. In build.xml, remove the Main-Class, which is incorrectly
  2995. set in tools, and rename the target "tools-jar" to "tools". (szetszwo)
  2996. MAPREDUCE-773. Sends progress reports for compressed gzip inputs in maps.
  2997. Fixes a native direct buffer leak in LineRecordReader classes.
  2998. (Hong Tang and ddas)
  2999. MAPREDUCE-832. Reduce number of warning messages printed when
  3000. deprecated memory variables are used. (Rahul Kumar Singh via yhemanth)
  3001. MAPREDUCE-745. Fixes a testcase problem to do with generation of JobTracker
  3002. IDs. (Amar Kamat via ddas)
  3003. MAPREDUCE-834. Enables memory management on tasktrackers when old
  3004. memory management parameters are used in configuration.
  3005. (Sreekanth Ramakrishnan via yhemanth)
  3006. MAPREDUCE-818. Fixes Counters#getGroup API. (Amareshwari Sriramadasu
  3007. via sharad)
  3008. MAPREDUCE-807. Handles the AccessControlException during the deletion of
  3009. mapred.system.dir in the JobTracker. The JobTracker will bail out if it
  3010. encounters such an exception. (Amar Kamat via ddas)
  3011. MAPREDUCE-430. Fix a bug related to task getting stuck in case of
  3012. OOM error. (Amar Kamat via ddas)
  3013. MAPREDUCE-871. Fix ownership of Job/Task local files to have correct
  3014. group ownership according to the egid of the tasktracker.
  3015. (Vinod Kumar Vavilapalli via yhemanth)
  3016. MAPREDUCE-911. Fix a bug in TestTaskFail related to speculative
  3017. execution. (Amareshwari Sriramadasu via sharad)
  3018. MAPREDUCE-687. Fix an assertion in TestMiniMRMapRedDebugScript.
  3019. (Amareshwari Sriramadasu via sharad)
  3020. MAPREDUCE-924. Fixes the TestPipes testcase to use Tool.
  3021. (Amareshwari Sriramadasu via sharad)
  3022. MAPREDUCE-903. Add Avro jar to eclipse classpath.
  3023. (Philip Zeyliger via tomwhite)
  3024. MAPREDUCE-943. Removes a testcase in TestNodeRefresh that doesn't make
  3025. sense in the new Job recovery model. (Amar Kamat via ddas)
  3026. MAPREDUCE-764. TypedBytesInput's readRaw() does not preserve custom type
  3027. codes. (Klaas Bosteels via tomwhite)
  3028. HADOOP-6243. Fixes a NullPointerException in handling deprecated keys.
  3029. (Sreekanth Ramakrishnan via yhemanth)
  3030. MAPREDUCE-968. NPE in distcp encountered when placing _logs directory on
  3031. S3FileSystem. (Aaron Kimball via tomwhite)
  3032. MAPREDUCE-826. harchive doesn't use ToolRunner / harchive returns 0 even
  3033. if the job fails with exception (koji Noguchi via mahadev)
  3034. MAPREDUCE-839. unit test TestMiniMRChildTask fails on mac os-x (hong tang
  3035. via mahadev)
  3036. MAPREDUCE-112. Add counters for reduce input, output records to the new API.
  3037. (Jothi Padmanabhan via cdouglas)
  3038. MAPREDUCE-648. Fix two distcp bugs: (1) it should not launch a job if all
  3039. src paths are directories, and (2) it does not skip copying when updating
  3040. a single file. (Ravi Gummadi via szetszwo)
  3041. MAPREDUCE-946. Fix a regression in LineRecordReader where the
  3042. maxBytesToConsume parameter is not set correctly. (cdouglas)
  3043. MAPREDUCE-977. Missing jackson jars from Eclipse template. (tomwhite)
  3044. MAPREDUCE-988. Fix a packaging issue in the contrib modules. (Hong Tang via
  3045. cdouglas)
  3046. MAPREDUCE-971. distcp does not always remove distcp.tmp.dir. (Aaron Kimball
  3047. via tomwhite)
  3048. MAPREDUCE-995. Fix a bug in JobHistory where tasks completing after the job
  3049. is closed cause a NPE. (Jothi Padmanabhan via cdouglas)
  3050. MAPREDUCE-953. Fix QueueManager to dump queue configuration in JSON format.
  3051. (V.V. Chaitanya Krishna via yhemanth)
  3052. MAPREDUCE-645. Prevent distcp from running a job when the destination is a
  3053. file, but the source is not. (Ravi Gummadi via cdouglas)
  3054. MAPREDUCE-1002. Flushed writer in JobQueueClient so queue information is
  3055. printed correctly. (V.V. Chaitanya Krishna via yhemanth)
  3056. MAPREDUCE-1003. Fix compilation problem in eclipse plugin when
  3057. eclipse.home is set. (Ravi Gummadi via yhemanth)
  3058. MAPREDUCE-941. Vaidya script fails on Solaris. (Chad Metcalf
  3059. via tomwhite)
  3060. MAPREDUCE-912. Add and standardize Apache license headers. (Chad Metcalf
  3061. via cdouglas)
  3062. MAPREDUCE-1022. Fix compilation of vertica testcases. (Vinod Kumar
  3063. Vavilapalli via acmurthy)
  3064. MAPREDUCE-1000. Handle corrupt history files in JobHistory.initDone().
  3065. (Jothi Padmanabhan via sharad)
  3066. MAPREDUCE-1028. Fixed number of slots occupied by cleanup tasks to one
  3067. irrespective of slot size for the job.
  3068. (Ravi Gummadi via yhemanth)
  3069. MAPREDUCE-964. Fixed start and finish times of TaskStatus to be
  3070. consistent, thereby fixing inconsistencies in metering tasks.
  3071. (Sreekanth Ramakrishnan via yhemanth)
  3072. MAPREDUCE-1076. Deprecate ClusterStatus and add javadoc in ClusterMetrics.
  3073. (Amareshwari Sriramadasu via sharad)
  3074. MAPREDUCE-979. Fixed JobConf APIs related to memory parameters to return
  3075. values of new configuration variables when deprecated variables are
  3076. disabled. (Sreekanth Ramakrishnan via yhemanth)
  3077. MAPREDUCE-1030. Modified scheduling algorithm to return a map and reduce
  3078. task per heartbeat in the capacity scheduler.
  3079. (Rahul Kumar Singh via yhemanth)
  3080. MAPREDUCE-1071. Use DataInputStream rather than FSDataInputStream in the
  3081. JobHistory EventReader. (Hong Tang via cdouglas)
  3082. MAPREDUCE-986. Fix Rumen to work with truncated task lines. (Dick King via
  3083. cdouglas)
  3084. MAPREDUCE-1029. Fix failing TestCopyFiles by restoring the unzipping of
  3085. HDFS webapps from the hdfs jar. (Aaron Kimball and Jothi Padmanabhan via
  3086. cdouglas)
  3087. MAPREDUCE-769. Make findbugs and javac warnings to zero.
  3088. (Amareshwari Sriramadasu via sharad)
  3089. MAPREDUCE-1104. Initialize RecoveryManager in JobTracker cstr called by
  3090. Mumak. (Hong Tang via cdouglas)
  3091. MAPREDUCE-1061. Add unit test validating byte specifications for gridmix
  3092. jobs. (cdouglas)
  3093. MAPREDUCE-1077. Fix Rumen so that truncated tasks do not mark the job as
  3094. successful. (Dick King via cdouglas)
  3095. MAPREDUCE-1041. Make TaskInProgress::taskStatuses map package-private.
  3096. (Jothi Padmanabhan via cdouglas)
  3097. MAPREDUCE-1070. Prevent a deadlock in the fair scheduler servlet.
  3098. (Todd Lipcon via cdouglas)
  3099. MAPREDUCE-1086. Setup Hadoop logging environment for tasks to point to
  3100. task related parameters. (Ravi Gummadi via yhemanth)
  3101. MAPREDUCE-1105. Remove max limit configuration in capacity scheduler in
  3102. favor of max capacity percentage thus allowing the limit to go over
  3103. queue capacity. (Rahul Kumar Singh via yhemanth)
  3104. MAPREDUCE-1016. Make the job history log format JSON. (cutting)
  3105. MAPREDUCE-1038. Weave Mumak aspects only if related files have changed.
  3106. (Aaron Kimball via cdouglas)
  3107. MAPREDUCE-1163. Remove unused, hard-coded paths from libhdfs. (Allen
  3108. Wittenauer via cdouglas)
  3109. MAPREDUCE-962. Fix a NullPointerException while killing task process
  3110. trees. (Ravi Gummadi via yhemanth)
  3111. MAPREDUCE-1177. Correct setup/cleanup inversion in
  3112. JobTracker::getTaskReports. (Vinod Kumar Vavilapalli via cdouglas)
  3113. MAPREDUCE-1178. Fix ClassCastException in MultipleInputs by adding
  3114. a DelegatingRecordReader. (Amareshwari Sriramadasu and Jay Booth
  3115. via sharad)
  3116. MAPREDUCE-1068. Fix streaming job to show proper message if file is
  3117. is not present. (Amareshwari Sriramadasu via sharad)
  3118. MAPREDUCE-1147. Add map output counters to new API. (Amar Kamat via
  3119. cdouglas)
  3120. MAPREDUCE-915. The debug scripts are run as the job user. (ddas)
  3121. MAPREDUCE-1007. Fix NPE in CapacityTaskScheduler.getJobs().
  3122. (V.V.Chaitanya Krishna via sharad)
  3123. MAPREDUCE-28. Refactor TestQueueManager and fix default ACLs.
  3124. (V.V.Chaitanya Krishna and Rahul K Singh via sharad)
  3125. MAPREDUCE-1182. Fix overflow in reduce causing allocations to exceed the
  3126. configured threshold. (cdouglas)
  3127. MAPREDUCE-1239. Fix contrib components build dependencies.
  3128. (Giridharan Kesavan and omalley)
  3129. MAPREDUCE-787. Fix JobSubmitter to honor user given symlink path.
  3130. (Amareshwari Sriramadasu via sharad)
  3131. MAPREDUCE-1249. Update config default value for socket read timeout to
  3132. match code default. (Amareshwari Sriramadasu via cdouglas)
  3133. MAPREDUCE-1161. Remove ineffective synchronization in NotificationTestCase.
  3134. (Owen O'Malley via cdouglas)
  3135. MAPREDUCE-1244. Fix eclipse-plugin's build dependencies. (gkesavan)
  3136. MAPREDUCE-1075. Fix JobTracker to not throw an NPE for a non-existent
  3137. queue. (V.V.Chaitanya Krishna via yhemanth)
  3138. MAPREDUCE-754. Fix NPE in expiry thread when a TT is lost. (Amar Kamat
  3139. via sharad)
  3140. MAPREDUCE-1074. Document Reducer mark/reset functionality. (Jothi
  3141. Padmanabhan via cdouglas)
  3142. MAPREDUCE-1267. Fix typo in mapred-default.xml. (Todd Lipcon via cdouglas)
  3143. MAPREDUCE-952. Remove inadvertently reintroduced Task.Counter enum. (Jothi
  3144. Padmanabhan via cdouglas)
  3145. MAPREDUCE-1230. Fix handling of null records in VerticaInputFormat. (Omer
  3146. Trajman via cdouglas)
  3147. MAPREDUCE-1171. Allow shuffle retries and read-error reporting to be
  3148. configurable. (Amareshwari Sriramadasu via acmurthy)
  3149. MAPREDUCE-879. Fix broken unit test TestTaskTrackerLocalization on MacOS.
  3150. (Sreekanth Ramakrishnan via yhemanth)
  3151. MAPREDUCE-1124. Fix imprecise byte counts in Gridmix. (cdouglas)
  3152. MAPREDUCE-1222. Add an option to exclude numeric IP addresses in topologies
  3153. processed by Mumak. (Hong Tang via cdouglas)
  3154. MAPREDUCE-1284. Fix fts_open() call in task-controller that was failing
  3155. LinuxTaskController unit tests. (Ravi Gummadi via yhemanth)
  3156. MAPREDUCE-1143. Fix running task counters to be updated correctly
  3157. when speculative attempts are running for a TIP.
  3158. (Rahul Kumar Singh via yhemanth)
  3159. MAPREDUCE-1241. Use a default queue configuration in JobTracker when
  3160. mapred-queues.xml is unavailable. (Todd Lipcon via cdouglas)
  3161. MAPREDUCE-1301. Fix set up of permission checking script used in
  3162. localization tests. (Amareshwari Sriramadasu via yhemanth)
  3163. MAPREDUCE-1286. Remove quoting from client opts in TaskRunner. (Yuri
  3164. Pradkin via cdouglas)
  3165. MAPREDUCE-1059. Use distcp.bytes.per.map when adding sync markers in
  3166. distcp. (Aaron Kimball via cdouglas)
  3167. MAPREDUCE-1009. Update forrest documentation describing hierarchical
  3168. queues. (Vinod Kumar Vavilapalli via yhemanth)
  3169. MAPREDUCE-1342. Fixed deadlock in global blacklisting of tasktrackers.
  3170. (Amareshwari Sriramadasu via acmurthy)
  3171. MAPREDUCE-1316. Fixes a memory leak of TaskInProgress instances in
  3172. the jobtracker. (Amar Kamat via yhemanth)
  3173. MAPREDUCE-1359. TypedBytes TestIO doesn't mkdir its test dir first.
  3174. (Anatoli Fomenko via cos)
  3175. MAPREDUCE-1314. Correct errant mapreduce.x.mapreduce.x replacements from
  3176. bulk change. (Amareshwari Sriramadasu via cdouglas)
  3177. MAPREDUCE-1365. Restore accidentally renamed test in
  3178. TestTaskTrackerBloacklisting. (Amareshwari Sriramadasu via cdouglas)
  3179. MAPREDUCE-1406. Fix spelling of JobContext.MAP_COMBINE_MIN_SPILLS.
  3180. (cdouglas)
  3181. MAPREDUCE-1369. JUnit tests should never depend on anything in conf
  3182. (Anatoli Fomenko via cos)
  3183. MAPREDUCE-1412. Fix timer granularity issue causing failures in
  3184. TestTaskTrackerBlacklisting. (cdouglas)
  3185. MAPREDUCE-1448. Respect --config option in Mumak script. (Hong Tang via
  3186. cdouglas)
  3187. MAPREDUCE-1251. c++ utils doesn't compile. (Eli Collins via tomwhite)
  3188. MAPREDUCE-1522. FileInputFormat may use the default FileSystem for the
  3189. input path. (Tsz Wo (Nicholas), SZE via cdouglas)
  3190. MAPREDUCE-1407. Update javadoc in mapreduce.{Mapper,Reducer} to match
  3191. actual usage. (Benoit Sigoure via cdouglas)
  3192. MAPREDUCE-1258. Fix fair scheduler event log not logging job info.
  3193. (matei)
  3194. MAPREDUCE-1089. Fix NPE in fair scheduler preemption when tasks are
  3195. scheduled but not running. (Todd Lipcon via matei)
  3196. MAPREDUCE-1014. Fix the libraries for common and hdfs. (omalley)
  3197. MAPREDUCE-1111. JT Jetty UI not working if we run mumak.sh
  3198. off packaged distribution directory. (hong tang via mahadev)
  3199. MAPREDUCE-1133. Eclipse .classpath template has outdated jar files and is
  3200. missing some new ones. (cos)
  3201. MAPREDUCE-1098. Fixed the distributed-cache to not do i/o while holding a
  3202. global lock. (Amareshwari Sriramadasu via acmurthy)
  3203. MAPREDUCE-1158. Fix JT running maps and running reduces metrics.
  3204. (sharad)
  3205. MAPREDUCE-1160. Reduce verbosity of log lines in some Map/Reduce classes
  3206. to avoid filling up jobtracker logs on a busy cluster.
  3207. (Ravi Gummadi and Hong Tang via yhemanth)
  3208. MAPREDUCE-1153. Fix tasktracker metrics when trackers are decommissioned.
  3209. (sharad)
  3210. MAPREDUCE-1128. Fix MRUnit to prohibit iterating over values twice. (Aaron
  3211. Kimball via cdouglas)
  3212. MAPREDUCE-665. Move libhdfs to HDFS subproject. (Eli Collins via dhruba)
  3213. MAPREDUCE-1196. Fix FileOutputCommitter to use the deprecated cleanupJob
  3214. api correctly. (acmurthy)
  3215. MAPREDUCE-1244. Fix eclipse-plugin's build dependencies. (gkesavan)
  3216. MAPREDUCE-1140. Fix DistributedCache to not decrement reference counts for
  3217. unreferenced files in error conditions.
  3218. (Amareshwari Sriramadasu via yhemanth)
  3219. MAPREDUCE-1245. Fix TestFairScheduler failures by instantiating lightweight
  3220. Jobtracker. (sharad)
  3221. MAPREDUCE-1260. Update Eclipse configuration to match changes to Ivy
  3222. configuration. (Edwin Chan via cos)
  3223. MAPREDUCE-1152. Distinguish between failed and killed tasks in
  3224. JobTrackerInstrumentation. (Sharad Agarwal via cdouglas)
  3225. MAPREDUCE-1285. In DistCp.deleteNonexisting(..), get class from the
  3226. parameter instead of using FileStatus.class. (Peter Romianowski via
  3227. szetszwo)
  3228. MAPREDUCE-1294. Build fails to pull latest hadoop-core-* artifacts (cos)
  3229. MAPREDUCE-1213. TaskTrackers restart is faster because it deletes
  3230. distributed cache directory asynchronously. (Zheng Shao via dhruba)
  3231. MAPREDUCE-1265. The task attempt error log prints the name of the
  3232. tasktracker machine. (Scott Chen via dhruba)
  3233. MAPREDUCE-1201. ProcfsBasedProcessTree collects CPU usage information.
  3234. (Scott Chen via dhruba)
  3235. MAPREDUCE-1326. fi tests don't use fi-site.xml (cos)
  3236. MAPREDUCE-1165. Replace non-portable function name with C99 equivalent.
  3237. (Allen Wittenauer via cdouglas)
  3238. MAPREDUCE-1331. Fixes a typo in a testcase (Devaraj Das)
  3239. MAPREDUCE-1293. AutoInputFormat doesn't work with non-default FileSystems.
  3240. (Andrew Hitchcock via tomwhite)
  3241. MAPREDUCE-1131. Using profilers other than hprof can cause JobClient to
  3242. report job failure. (Aaron Kimball via tomwhite)
  3243. MAPREDUCE-1155. Streaming tests swallow exceptions.
  3244. (Todd Lipcon via tomwhite)
  3245. MAPREDUCE-1212. Mapreduce contrib project ivy dependencies are not included
  3246. in binary target. (Aaron Kimball via tomwhite)
  3247. MAPREDUCE-1388. Move the HDFS RAID package from HDFS to MAPREDUCE.
  3248. (Eli Collins via dhruba)
  3249. MAPREDUCE-1322. Defines default value for staging directory to be user
  3250. based fixing a failing streaming test.
  3251. (Devaraj Das and Amareshwari Sriramadasu via yhemanth)
  3252. MAPREDUCE-899. Modified LinuxTaskController to check that task-controller
  3253. has right permissions and ownership before performing any actions.
  3254. (Amareshwari Sriramadasu via yhemanth)
  3255. MAPREDUCE-1443. DBInputFormat can leak connections.
  3256. (Aaron Kimball via tomwhite)
  3257. MAPREDUCE-1457. Fixes JobTracker to get the FileSystem object within
  3258. getStagingAreaDir within a privileged block. Fixes Child.java to use the
  3259. appropriate UGIs while getting the TaskUmbilicalProtocol proxy and
  3260. while executing the task. (Jakob Homan via ddas)
  3261. MAPREDUCE-1399. The archive command shows a null error message (nicholas
  3262. via mahadev)
  3263. MAPREDUCE-1305. Improve efficiency of distcp -delete. (Peter Romianowski
  3264. via cdouglas)
  3265. MAPREDUCE-1474. Update forrest documentation for Hadoop Archives. (Mahadev
  3266. Konar via cdouglas)
  3267. MAPREDUCE-1400. Use tr rather than sed to effect literal substitution in
  3268. the build script. (Allen Wittenauer via cdouglas)
  3269. MAPREDUCE-1358. Avoid false positives in OutputLogFilter. (Todd Lipcon via
  3270. cdouglas)
  3271. MAPREDUCE-1490. Fix a NullPointerException that could occur during
  3272. instantiation and initialization of the DistributedRaidFileSystem.
  3273. (Rodrigo Schmidt via dhruba)
  3274. MAPREDUCE-1476. Fix the M/R framework to not call commit for special
  3275. tasks like job setup/cleanup and task cleanup.
  3276. (Amareshwari Sriramadasu via yhemanth)
  3277. MAPREDUCE-1398. Fix TaskLauncher to stop waiting for slots on a TIP that
  3278. is killed / failed.
  3279. (Amareshwari Sriramadasu via yhemanth)
  3280. MAPREDUCE-1491. The parity files created by the RAID are combined
  3281. using Hadoop Archive Files (HAR). (Rodrigo Schmidt via dhruba)
  3282. MAPREDUCE-1378. URL encode link in jobhistory.jsp to avoid errors caused by
  3283. unescaped characters. (E. Sammer via cdouglas)
  3284. MAPREDUCE-1519. RaidNode fails to create new parity file
  3285. if an older version already exists. (Rodrigo Schmidt via dhruba)
  3286. MAPREDUCE-1537. Fixes a compilation problem in a testcase after commit
  3287. HDFS-984. (Jitendra Nath Pandey via ddas)
  3288. MAPREDUCE-1537. The patch makes the job client call the getDelegationToken
  3289. only when security is enabled. (Jitendra Nath Pandey via ddas)
  3290. MAPREDUCE-1510. RAID should regenerate parity files if they get deleted.
  3291. (Rodrigo Schmidt via dhruba)
  3292. MAPREDUCE-1421. Fix the LinuxTaskController tests failing on trunk after
  3293. the commit of MAPREDUCE-1385. (Amareshwari Sriramadasu via vinodkv)
  3294. MAPREDUCE-1520. Fix TestMiniMRLocalFS failure caused by regression in
  3295. getting user working dir. (Amareshwari Sriramadasu via cdouglas)
  3296. MAPREDUCE-1512. RAID uses HarFileSystem directly instead of
  3297. FileSystem.get (Rodrigo Schmidt via dhruba)
  3298. MAPREDUCE-1435. Fix symlink handling in task work directory when
  3299. cleaning up, essentially to avoid following links.
  3300. (Ravi Gummadi via yhemanth)
  3301. MAPREDUCE-1518. RaidNode does not run the deletion check on the
  3302. directory that stores the parity files. (Rodrigo Schmidt via dhruba)
  3303. MAPREDUCE-1573. TestStreamingAsDifferentUser fails if run as tt_user.
  3304. (Ravi Gummadi via vinodkv)
  3305. MAPREDUCE-927. Cleanup of task-logs should happen in TaskTracker instead
  3306. of the Child. (Amareshwari Sriramadasu via vinodkv)
  3307. MAPREDUCE-1578. Decouple HadoopArchives vesrion from HarFileSystem version.
  3308. (Rodrigo Schmidt via szetszwo)
  3309. MAPREDUCE-1422. Fix cleanup of localized job directory to work if files
  3310. with non-deletable permissions are created within it.
  3311. (Amar Kamat via yhemanth)
  3312. MAPREDUCE-1306. Randomize the arrival of heartbeat responses in Mumak.
  3313. (Tamas Sarlos via cdouglas)
  3314. MAPREDUCE-1579. archive: check and possibly replace the space charater
  3315. in source paths. (szetszwo)
  3316. MAPREDUCE-1536. DataDrivenDBInputFormat does not split date columns correctly.
  3317. (Aaron Kimball via enis)
  3318. MAPREDUCE-890. After HADOOP-4491, the user who started mapred system is
  3319. not able to run job. (Ravi Gummadi via vinodkv)
  3320. MAPREDUCE-1615. Fix compilation of TestSubmitJob. (cdouglas)
  3321. MAPREDUCE-1508. Protect against NPE in TestMultipleLevelCaching. (Aaron
  3322. Kimball via cdouglas)
  3323. MAPREDUCE-1497. Suppress spurious findbugs warning about IndexCache
  3324. synchronization. (Amareshwari Sriramadasu via cdouglas)
  3325. MAPREDUCE-1420. Fix TestTTResourceReporting failure. (Scott Chen via
  3326. cdouglas)
  3327. MAPREDUCE-1480. Correctly initialize child RecordReaders in
  3328. CombineFileInputFormat. (Aaron Kimball via cdouglas)
  3329. MAPREDUCE-1348. Fix block forensics packaging. (Tom White via cdouglas)
  3330. MAPREDUCE-1628. HarFileSystem shows incorrect replication numbers and
  3331. permissions. (szetszwo via mahadev)
  3332. MAPREDUCE-1602. Fix the error message for the case that src does not
  3333. exist. (szetszwo)
  3334. MAPREDUCE-1585. Create Hadoop Archives version 2 with filenames
  3335. URL-encoded (rodrigo via mahadev)
  3336. MAPREDUCE-1523. Sometimes rumen trace generator fails to extract the job
  3337. finish time. (dick king via mahadev)
  3338. MAPREDUCE-1635. ResourceEstimator does not work after MAPREDUCE-842.
  3339. (Amareshwari Sriramadasu via vinodkv)
  3340. MAPREDUCE-889. binary communication formats added to Streaming by
  3341. HADOOP-1722 should be documented. (Klaas Bosteels via tomwhite)
  3342. MAPREDUCE-1031. ant tar target doens't seem to compile tests in contrib
  3343. projects. (Aaron Kimball via tomwhite)
  3344. MAPREDUCE-1692. Removed unused testcase TestStreamedMerge.
  3345. (Sreekanth Ramakrishnan and Amareshwari Sriramadasu via yhemanth)
  3346. MAPREDUCE-1062. Fix ReliabilityTest to work with retired jobs. (Sreekanth
  3347. Ramakrishnan via cdouglas)
  3348. MAPREDUCE-1409. IOExceptions thrown from FIleOutputCommitter::abortTask
  3349. should cause the task to fail. (Amareshwari Sriramadasu via cdouglas)
  3350. MAPREDUCE-1695. Include capacity scheduler in findbugs and javadoc-dev
  3351. targets and also fix existing warnings. (Hong Tang via yhemanth)
  3352. MAPREDUCE-1494. Ensure TestJobDirCleanup verifies the correct paths.
  3353. (Amareshwari Sriramadasu via cdouglas)
  3354. MAPREDUCE-1622. Include missing slf4j dependencies. (cdouglas)
  3355. MAPREDUCE-1515. Accept java5.home from build.properties, not only from the
  3356. command line when building forrest docs. (Al Thompson via cdouglas)
  3357. MAPREDUCE-1618. Add missing javadoc to JobStatus::*JobAcls. (Amareshwari
  3358. Sriramadasu via cdouglas)
  3359. MAPREDUCE-1219. Remove job level metrics from jobtracker metrics to ease
  3360. undue load on jobtracker. (Sreekanth Ramakrishnan via sharad)
  3361. MAPREDUCE-1604. Add Forrest documentation for Job ACLs.
  3362. (Amareshwari Sriramadasu via yhemanth)
  3363. MAPREDUCE-1705. Archiving and Purging of HDFS parity files should
  3364. handle globbed policies accurately. (Rodrigo Schmidt via dhruba)
  3365. MAPREDUCE-1612. job conf file is not accessible from job history web page.
  3366. (Ravi Gummadi and Sreekanth Ramakrishnan via vinodkv)
  3367. MAPREDUCE-1397. NullPointerException observed during task failures.
  3368. (Amareshwari Sriramadasu via vinodkv)
  3369. MAPREDUCE-1728. Oracle timezone strings do not match Java.
  3370. (Aaron Kimball via tomwhite)
  3371. MAPREDUCE-1609. TaskTracker.localizeJob should not set permissions on
  3372. job log directory recursively. (Amareshwari Sriramadasu via vinodkv)
  3373. MAPREDUCE-1657. After task logs directory is deleted, tasklog servlet
  3374. displays wrong error message about job ACLs. (Ravi Gummadi via vinodkv)
  3375. MAPREDUCE-1727. TestJobACLs fails after HADOOP-6686. (Ravi Gummadi via vinodkv)
  3376. MAPREDUCE-1611. Refresh nodes and refresh queues doesnt work with service
  3377. authorization enabled. (Amar Kamat via vinodkv)
  3378. MAPREDUCE-1276. Correct flaws in the shuffle related to connection setup
  3379. and failure attribution. (Amareshwari Sriramadasu via cdouglas)
  3380. MAPREDUCE-1372. ConcurrentModificationException in JobInProgress.
  3381. (Dick King and Amareshwari Sriramadasu via tomwhite)
  3382. MAPREDUCE-118. Fix Job.getJobID(). (Amareshwari Sriramadasu via sharad)
  3383. MAPREDUCE-913. TaskRunner crashes with NPE resulting in held up slots,
  3384. UNINITIALIZED tasks and hung TaskTracker. (Amareshwari Sriramadasu and
  3385. Sreekanth Ramakrishnan via vinodkv)
  3386. MAPREDUCE-1725. Fix MapReduce API incompatibilities between 0.20 and 0.21.
  3387. (tomwhite)
  3388. MAPREDUCE-1606. TestJobACLs may timeout as there are no slots for launching
  3389. JOB_CLEANUP task. (Ravi Gummadi via vinodkv)
  3390. MAPREDUCE-1765. Correct streaming documentation for StreamXmlRecordReader.
  3391. (Corinne Chandel via amareshwari)
  3392. MAPREDUCE-1880. Fix BigDecimal.divide(..) in the pi example. (szetszwo)
  3393. MAPREDUCE-1885. Revert FileSystem create method that takes CreateFlags
  3394. (MapReduce part of HADOOP-6826). (Ravi Gummadi via tomwhite)
  3395. MAPREDUCE-1870. Harmonize MapReduce JAR library versions with Common and
  3396. HDFS. (tomwhite)
  3397. MAPREDUCE-1791. Remote cluster control functionality needs JavaDocs
  3398. improvement (Konstantin Boudnik)
  3399. MAPREDUCE-1942. 'compile-fault-inject' should never be called directly.
  3400. (Konstantin Boudnik)
  3401. MAPREDUCE-1876. Fixes TaskAttemptStartedEvent to correctly log event type
  3402. for all task types. (Amar Kamat via amareshwari)
  3403. MAPREDUCE-1926. MapReduce distribution is missing build-utils.xml.
  3404. (tomwhite)
  3405. MAPREDUCE-2012. Some contrib tests fail in branch 0.21 and trunk.
  3406. (Amareshwari Sriramadasu via tomwhite)
  3407. MAPREDUCE-1980. Fixes TaskAttemptUnsuccessfulCompletionEvent and
  3408. TaskAttemptFinishedEvent to correctly log event type for all task types.
  3409. (Amar Kamat via amareshwari)
  3410. MAPREDUCE-1856. Extract a subset of tests for smoke (DOA) validation (cos)