CHANGES.txt 310 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197119811991200120112021203120412051206120712081209121012111212121312141215121612171218121912201221122212231224122512261227122812291230123112321233123412351236123712381239124012411242124312441245124612471248124912501251125212531254125512561257125812591260126112621263126412651266126712681269127012711272127312741275127612771278127912801281128212831284128512861287128812891290129112921293129412951296129712981299130013011302130313041305130613071308130913101311131213131314131513161317131813191320132113221323132413251326132713281329133013311332133313341335133613371338133913401341134213431344134513461347134813491350135113521353135413551356135713581359136013611362136313641365136613671368136913701371137213731374137513761377137813791380138113821383138413851386138713881389139013911392139313941395139613971398139914001401140214031404140514061407140814091410141114121413141414151416141714181419142014211422142314241425142614271428142914301431143214331434143514361437143814391440144114421443144414451446144714481449145014511452145314541455145614571458145914601461146214631464146514661467146814691470147114721473147414751476147714781479148014811482148314841485148614871488148914901491149214931494149514961497149814991500150115021503150415051506150715081509151015111512151315141515151615171518151915201521152215231524152515261527152815291530153115321533153415351536153715381539154015411542154315441545154615471548154915501551155215531554155515561557155815591560156115621563156415651566156715681569157015711572157315741575157615771578157915801581158215831584158515861587158815891590159115921593159415951596159715981599160016011602160316041605160616071608160916101611161216131614161516161617161816191620162116221623162416251626162716281629163016311632163316341635163616371638163916401641164216431644164516461647164816491650165116521653165416551656165716581659166016611662166316641665166616671668166916701671167216731674167516761677167816791680168116821683168416851686168716881689169016911692169316941695169616971698169917001701170217031704170517061707170817091710171117121713171417151716171717181719172017211722172317241725172617271728172917301731173217331734173517361737173817391740174117421743174417451746174717481749175017511752175317541755175617571758175917601761176217631764176517661767176817691770177117721773177417751776177717781779178017811782178317841785178617871788178917901791179217931794179517961797179817991800180118021803180418051806180718081809181018111812181318141815181618171818181918201821182218231824182518261827182818291830183118321833183418351836183718381839184018411842184318441845184618471848184918501851185218531854185518561857185818591860186118621863186418651866186718681869187018711872187318741875187618771878187918801881188218831884188518861887188818891890189118921893189418951896189718981899190019011902190319041905190619071908190919101911191219131914191519161917191819191920192119221923192419251926192719281929193019311932193319341935193619371938193919401941194219431944194519461947194819491950195119521953195419551956195719581959196019611962196319641965196619671968196919701971197219731974197519761977197819791980198119821983198419851986198719881989199019911992199319941995199619971998199920002001200220032004200520062007200820092010201120122013201420152016201720182019202020212022202320242025202620272028202920302031203220332034203520362037203820392040204120422043204420452046204720482049205020512052205320542055205620572058205920602061206220632064206520662067206820692070207120722073207420752076207720782079208020812082208320842085208620872088208920902091209220932094209520962097209820992100210121022103210421052106210721082109211021112112211321142115211621172118211921202121212221232124212521262127212821292130213121322133213421352136213721382139214021412142214321442145214621472148214921502151215221532154215521562157215821592160216121622163216421652166216721682169217021712172217321742175217621772178217921802181218221832184218521862187218821892190219121922193219421952196219721982199220022012202220322042205220622072208220922102211221222132214221522162217221822192220222122222223222422252226222722282229223022312232223322342235223622372238223922402241224222432244224522462247224822492250225122522253225422552256225722582259226022612262226322642265226622672268226922702271227222732274227522762277227822792280228122822283228422852286228722882289229022912292229322942295229622972298229923002301230223032304230523062307230823092310231123122313231423152316231723182319232023212322232323242325232623272328232923302331233223332334233523362337233823392340234123422343234423452346234723482349235023512352235323542355235623572358235923602361236223632364236523662367236823692370237123722373237423752376237723782379238023812382238323842385238623872388238923902391239223932394239523962397239823992400240124022403240424052406240724082409241024112412241324142415241624172418241924202421242224232424242524262427242824292430243124322433243424352436243724382439244024412442244324442445244624472448244924502451245224532454245524562457245824592460246124622463246424652466246724682469247024712472247324742475247624772478247924802481248224832484248524862487248824892490249124922493249424952496249724982499250025012502250325042505250625072508250925102511251225132514251525162517251825192520252125222523252425252526252725282529253025312532253325342535253625372538253925402541254225432544254525462547254825492550255125522553255425552556255725582559256025612562256325642565256625672568256925702571257225732574257525762577257825792580258125822583258425852586258725882589259025912592259325942595259625972598259926002601260226032604260526062607260826092610261126122613261426152616261726182619262026212622262326242625262626272628262926302631263226332634263526362637263826392640264126422643264426452646264726482649265026512652265326542655265626572658265926602661266226632664266526662667266826692670267126722673267426752676267726782679268026812682268326842685268626872688268926902691269226932694269526962697269826992700270127022703270427052706270727082709271027112712271327142715271627172718271927202721272227232724272527262727272827292730273127322733273427352736273727382739274027412742274327442745274627472748274927502751275227532754275527562757275827592760276127622763276427652766276727682769277027712772277327742775277627772778277927802781278227832784278527862787278827892790279127922793279427952796279727982799280028012802280328042805280628072808280928102811281228132814281528162817281828192820282128222823282428252826282728282829283028312832283328342835283628372838283928402841284228432844284528462847284828492850285128522853285428552856285728582859286028612862286328642865286628672868286928702871287228732874287528762877287828792880288128822883288428852886288728882889289028912892289328942895289628972898289929002901290229032904290529062907290829092910291129122913291429152916291729182919292029212922292329242925292629272928292929302931293229332934293529362937293829392940294129422943294429452946294729482949295029512952295329542955295629572958295929602961296229632964296529662967296829692970297129722973297429752976297729782979298029812982298329842985298629872988298929902991299229932994299529962997299829993000300130023003300430053006300730083009301030113012301330143015301630173018301930203021302230233024302530263027302830293030303130323033303430353036303730383039304030413042304330443045304630473048304930503051305230533054305530563057305830593060306130623063306430653066306730683069307030713072307330743075307630773078307930803081308230833084308530863087308830893090309130923093309430953096309730983099310031013102310331043105310631073108310931103111311231133114311531163117311831193120312131223123312431253126312731283129313031313132313331343135313631373138313931403141314231433144314531463147314831493150315131523153315431553156315731583159316031613162316331643165316631673168316931703171317231733174317531763177317831793180318131823183318431853186318731883189319031913192319331943195319631973198319932003201320232033204320532063207320832093210321132123213321432153216321732183219322032213222322332243225322632273228322932303231323232333234323532363237323832393240324132423243324432453246324732483249325032513252325332543255325632573258325932603261326232633264326532663267326832693270327132723273327432753276327732783279328032813282328332843285328632873288328932903291329232933294329532963297329832993300330133023303330433053306330733083309331033113312331333143315331633173318331933203321332233233324332533263327332833293330333133323333333433353336333733383339334033413342334333443345334633473348334933503351335233533354335533563357335833593360336133623363336433653366336733683369337033713372337333743375337633773378337933803381338233833384338533863387338833893390339133923393339433953396339733983399340034013402340334043405340634073408340934103411341234133414341534163417341834193420342134223423342434253426342734283429343034313432343334343435343634373438343934403441344234433444344534463447344834493450345134523453345434553456345734583459346034613462346334643465346634673468346934703471347234733474347534763477347834793480348134823483348434853486348734883489349034913492349334943495349634973498349935003501350235033504350535063507350835093510351135123513351435153516351735183519352035213522352335243525352635273528352935303531353235333534353535363537353835393540354135423543354435453546354735483549355035513552355335543555355635573558355935603561356235633564356535663567356835693570357135723573357435753576357735783579358035813582358335843585358635873588358935903591359235933594359535963597359835993600360136023603360436053606360736083609361036113612361336143615361636173618361936203621362236233624362536263627362836293630363136323633363436353636363736383639364036413642364336443645364636473648364936503651365236533654365536563657365836593660366136623663366436653666366736683669367036713672367336743675367636773678367936803681368236833684368536863687368836893690369136923693369436953696369736983699370037013702370337043705370637073708370937103711371237133714371537163717371837193720372137223723372437253726372737283729373037313732373337343735373637373738373937403741374237433744374537463747374837493750375137523753375437553756375737583759376037613762376337643765376637673768376937703771377237733774377537763777377837793780378137823783378437853786378737883789379037913792379337943795379637973798379938003801380238033804380538063807380838093810381138123813381438153816381738183819382038213822382338243825382638273828382938303831383238333834383538363837383838393840384138423843384438453846384738483849385038513852385338543855385638573858385938603861386238633864386538663867386838693870387138723873387438753876387738783879388038813882388338843885388638873888388938903891389238933894389538963897389838993900390139023903390439053906390739083909391039113912391339143915391639173918391939203921392239233924392539263927392839293930393139323933393439353936393739383939394039413942394339443945394639473948394939503951395239533954395539563957395839593960396139623963396439653966396739683969397039713972397339743975397639773978397939803981398239833984398539863987398839893990399139923993399439953996399739983999400040014002400340044005400640074008400940104011401240134014401540164017401840194020402140224023402440254026402740284029403040314032403340344035403640374038403940404041404240434044404540464047404840494050405140524053405440554056405740584059406040614062406340644065406640674068406940704071407240734074407540764077407840794080408140824083408440854086408740884089409040914092409340944095409640974098409941004101410241034104410541064107410841094110411141124113411441154116411741184119412041214122412341244125412641274128412941304131413241334134413541364137413841394140414141424143414441454146414741484149415041514152415341544155415641574158415941604161416241634164416541664167416841694170417141724173417441754176417741784179418041814182418341844185418641874188418941904191419241934194419541964197419841994200420142024203420442054206420742084209421042114212421342144215421642174218421942204221422242234224422542264227422842294230423142324233423442354236423742384239424042414242424342444245424642474248424942504251425242534254425542564257425842594260426142624263426442654266426742684269427042714272427342744275427642774278427942804281428242834284428542864287428842894290429142924293429442954296429742984299430043014302430343044305430643074308430943104311431243134314431543164317431843194320432143224323432443254326432743284329433043314332433343344335433643374338433943404341434243434344434543464347434843494350435143524353435443554356435743584359436043614362436343644365436643674368436943704371437243734374437543764377437843794380438143824383438443854386438743884389439043914392439343944395439643974398439944004401440244034404440544064407440844094410441144124413441444154416441744184419442044214422442344244425442644274428442944304431443244334434443544364437443844394440444144424443444444454446444744484449445044514452445344544455445644574458445944604461446244634464446544664467446844694470447144724473447444754476447744784479448044814482448344844485448644874488448944904491449244934494449544964497449844994500450145024503450445054506450745084509451045114512451345144515451645174518451945204521452245234524452545264527452845294530453145324533453445354536453745384539454045414542454345444545454645474548454945504551455245534554455545564557455845594560456145624563456445654566456745684569457045714572457345744575457645774578457945804581458245834584458545864587458845894590459145924593459445954596459745984599460046014602460346044605460646074608460946104611461246134614461546164617461846194620462146224623462446254626462746284629463046314632463346344635463646374638463946404641464246434644464546464647464846494650465146524653465446554656465746584659466046614662466346644665466646674668466946704671467246734674467546764677467846794680468146824683468446854686468746884689469046914692469346944695469646974698469947004701470247034704470547064707470847094710471147124713471447154716471747184719472047214722472347244725472647274728472947304731473247334734473547364737473847394740474147424743474447454746474747484749475047514752475347544755475647574758475947604761476247634764476547664767476847694770477147724773477447754776477747784779478047814782478347844785478647874788478947904791479247934794479547964797479847994800480148024803480448054806480748084809481048114812481348144815481648174818481948204821482248234824482548264827482848294830483148324833483448354836483748384839484048414842484348444845484648474848484948504851485248534854485548564857485848594860486148624863486448654866486748684869487048714872487348744875487648774878487948804881488248834884488548864887488848894890489148924893489448954896489748984899490049014902490349044905490649074908490949104911491249134914491549164917491849194920492149224923492449254926492749284929493049314932493349344935493649374938493949404941494249434944494549464947494849494950495149524953495449554956495749584959496049614962496349644965496649674968496949704971497249734974497549764977497849794980498149824983498449854986498749884989499049914992499349944995499649974998499950005001500250035004500550065007500850095010501150125013501450155016501750185019502050215022502350245025502650275028502950305031503250335034503550365037503850395040504150425043504450455046504750485049505050515052505350545055505650575058505950605061506250635064506550665067506850695070507150725073507450755076507750785079508050815082508350845085508650875088508950905091509250935094509550965097509850995100510151025103510451055106510751085109511051115112511351145115511651175118511951205121512251235124512551265127512851295130513151325133513451355136513751385139514051415142514351445145514651475148514951505151515251535154515551565157515851595160516151625163516451655166516751685169517051715172517351745175517651775178517951805181518251835184518551865187518851895190519151925193519451955196519751985199520052015202520352045205520652075208520952105211521252135214521552165217521852195220522152225223522452255226522752285229523052315232523352345235523652375238523952405241524252435244524552465247524852495250525152525253525452555256525752585259526052615262526352645265526652675268526952705271527252735274527552765277527852795280528152825283528452855286528752885289529052915292529352945295529652975298529953005301530253035304530553065307530853095310531153125313531453155316531753185319532053215322532353245325532653275328532953305331533253335334533553365337533853395340534153425343534453455346534753485349535053515352535353545355535653575358535953605361536253635364536553665367536853695370537153725373537453755376537753785379538053815382538353845385538653875388538953905391539253935394539553965397539853995400540154025403540454055406540754085409541054115412541354145415541654175418541954205421542254235424542554265427542854295430543154325433543454355436543754385439544054415442544354445445544654475448544954505451545254535454545554565457545854595460546154625463546454655466546754685469547054715472547354745475547654775478547954805481548254835484548554865487548854895490549154925493549454955496549754985499550055015502550355045505550655075508550955105511551255135514551555165517551855195520552155225523552455255526552755285529553055315532553355345535553655375538553955405541554255435544554555465547554855495550555155525553555455555556555755585559556055615562556355645565556655675568556955705571557255735574557555765577557855795580558155825583558455855586558755885589559055915592559355945595559655975598559956005601560256035604560556065607560856095610561156125613561456155616561756185619562056215622562356245625562656275628562956305631563256335634563556365637563856395640564156425643564456455646564756485649565056515652565356545655565656575658565956605661566256635664566556665667566856695670567156725673567456755676567756785679568056815682568356845685568656875688568956905691569256935694569556965697569856995700570157025703570457055706570757085709571057115712571357145715571657175718571957205721572257235724572557265727572857295730573157325733573457355736573757385739574057415742574357445745574657475748574957505751575257535754575557565757575857595760576157625763576457655766576757685769577057715772577357745775577657775778577957805781578257835784578557865787578857895790579157925793579457955796579757985799580058015802580358045805580658075808580958105811581258135814581558165817581858195820582158225823582458255826582758285829583058315832583358345835583658375838583958405841584258435844584558465847584858495850585158525853585458555856585758585859586058615862586358645865586658675868586958705871587258735874587558765877587858795880588158825883588458855886588758885889589058915892589358945895589658975898589959005901590259035904590559065907590859095910591159125913591459155916591759185919592059215922592359245925592659275928592959305931593259335934593559365937593859395940594159425943594459455946594759485949595059515952595359545955595659575958595959605961596259635964596559665967596859695970597159725973597459755976597759785979598059815982598359845985598659875988598959905991599259935994599559965997599859996000600160026003600460056006600760086009601060116012601360146015601660176018601960206021602260236024602560266027602860296030603160326033603460356036603760386039604060416042604360446045604660476048604960506051605260536054605560566057605860596060606160626063606460656066606760686069607060716072607360746075607660776078607960806081608260836084608560866087608860896090609160926093609460956096609760986099610061016102610361046105610661076108610961106111611261136114611561166117611861196120612161226123612461256126612761286129613061316132613361346135613661376138613961406141614261436144614561466147614861496150615161526153615461556156615761586159616061616162616361646165616661676168616961706171617261736174617561766177617861796180618161826183618461856186618761886189619061916192619361946195619661976198619962006201620262036204620562066207620862096210621162126213621462156216621762186219622062216222622362246225622662276228622962306231623262336234623562366237623862396240624162426243624462456246624762486249625062516252625362546255625662576258625962606261626262636264626562666267626862696270627162726273627462756276627762786279628062816282628362846285628662876288628962906291629262936294629562966297629862996300630163026303630463056306630763086309631063116312631363146315631663176318631963206321632263236324632563266327632863296330633163326333633463356336633763386339634063416342634363446345634663476348634963506351635263536354635563566357635863596360636163626363636463656366636763686369637063716372637363746375637663776378637963806381638263836384638563866387638863896390639163926393639463956396639763986399640064016402640364046405640664076408640964106411641264136414641564166417641864196420642164226423642464256426642764286429643064316432643364346435643664376438643964406441644264436444644564466447644864496450645164526453645464556456645764586459646064616462646364646465646664676468646964706471647264736474647564766477647864796480648164826483648464856486648764886489649064916492649364946495649664976498649965006501650265036504650565066507650865096510651165126513651465156516651765186519652065216522652365246525652665276528652965306531653265336534653565366537653865396540654165426543654465456546654765486549655065516552655365546555655665576558655965606561656265636564656565666567656865696570657165726573657465756576657765786579658065816582658365846585658665876588658965906591659265936594659565966597659865996600660166026603660466056606660766086609661066116612661366146615661666176618661966206621662266236624662566266627662866296630663166326633663466356636663766386639664066416642664366446645664666476648664966506651665266536654665566566657665866596660666166626663666466656666666766686669667066716672667366746675667666776678667966806681668266836684668566866687668866896690669166926693669466956696669766986699670067016702670367046705670667076708670967106711671267136714671567166717671867196720672167226723672467256726672767286729673067316732673367346735673667376738673967406741674267436744674567466747674867496750675167526753675467556756675767586759676067616762676367646765676667676768676967706771677267736774677567766777677867796780678167826783678467856786678767886789679067916792679367946795679667976798679968006801680268036804680568066807680868096810681168126813681468156816681768186819682068216822682368246825682668276828682968306831683268336834683568366837683868396840684168426843684468456846684768486849685068516852685368546855685668576858685968606861686268636864686568666867686868696870687168726873687468756876687768786879688068816882688368846885688668876888688968906891689268936894689568966897689868996900690169026903690469056906690769086909691069116912691369146915691669176918691969206921692269236924692569266927692869296930693169326933693469356936693769386939694069416942694369446945694669476948694969506951695269536954695569566957695869596960696169626963696469656966696769686969697069716972697369746975697669776978697969806981698269836984698569866987698869896990699169926993699469956996699769986999700070017002700370047005700670077008700970107011701270137014701570167017701870197020702170227023702470257026702770287029703070317032703370347035703670377038703970407041704270437044704570467047704870497050705170527053705470557056705770587059706070617062706370647065706670677068706970707071707270737074707570767077707870797080708170827083708470857086708770887089709070917092709370947095709670977098709971007101710271037104710571067107710871097110711171127113711471157116711771187119712071217122712371247125712671277128712971307131713271337134713571367137713871397140714171427143714471457146714771487149715071517152715371547155715671577158715971607161716271637164716571667167716871697170717171727173717471757176717771787179718071817182718371847185718671877188718971907191719271937194719571967197719871997200720172027203720472057206720772087209721072117212721372147215721672177218721972207221722272237224722572267227722872297230723172327233723472357236723772387239724072417242724372447245724672477248724972507251725272537254725572567257725872597260726172627263726472657266726772687269727072717272727372747275727672777278727972807281728272837284728572867287728872897290729172927293729472957296729772987299730073017302730373047305730673077308730973107311731273137314731573167317731873197320732173227323732473257326732773287329733073317332733373347335733673377338733973407341734273437344734573467347734873497350735173527353735473557356735773587359736073617362736373647365736673677368736973707371737273737374737573767377737873797380738173827383738473857386738773887389739073917392739373947395739673977398739974007401740274037404740574067407740874097410741174127413741474157416741774187419742074217422742374247425742674277428742974307431743274337434743574367437743874397440744174427443744474457446744774487449745074517452745374547455745674577458745974607461746274637464746574667467746874697470747174727473747474757476747774787479748074817482748374847485748674877488748974907491749274937494749574967497749874997500750175027503750475057506750775087509751075117512751375147515751675177518751975207521752275237524752575267527752875297530753175327533753475357536753775387539754075417542754375447545754675477548754975507551755275537554755575567557755875597560756175627563756475657566756775687569757075717572757375747575757675777578757975807581758275837584758575867587758875897590759175927593759475957596759775987599760076017602760376047605760676077608760976107611761276137614761576167617761876197620762176227623762476257626762776287629763076317632763376347635763676377638763976407641764276437644764576467647764876497650765176527653765476557656765776587659766076617662766376647665766676677668766976707671767276737674767576767677767876797680768176827683768476857686768776887689769076917692769376947695769676977698769977007701770277037704770577067707770877097710771177127713771477157716771777187719772077217722772377247725772677277728772977307731773277337734773577367737773877397740774177427743774477457746774777487749775077517752775377547755775677577758775977607761776277637764776577667767776877697770777177727773777477757776777777787779778077817782778377847785778677877788778977907791779277937794779577967797779877997800780178027803780478057806780778087809781078117812781378147815781678177818781978207821782278237824782578267827782878297830783178327833783478357836783778387839784078417842784378447845784678477848784978507851785278537854785578567857785878597860786178627863786478657866786778687869787078717872787378747875787678777878787978807881788278837884788578867887788878897890789178927893789478957896789778987899790079017902790379047905790679077908790979107911791279137914791579167917791879197920792179227923792479257926792779287929793079317932793379347935793679377938793979407941794279437944794579467947794879497950795179527953795479557956795779587959796079617962796379647965796679677968796979707971797279737974797579767977797879797980798179827983798479857986798779887989799079917992799379947995799679977998799980008001800280038004800580068007800880098010801180128013801480158016801780188019802080218022802380248025802680278028802980308031803280338034803580368037803880398040804180428043804480458046804780488049805080518052805380548055805680578058805980608061806280638064806580668067806880698070807180728073807480758076807780788079808080818082808380848085808680878088808980908091809280938094809580968097809880998100810181028103810481058106810781088109811081118112811381148115811681178118811981208121812281238124812581268127812881298130813181328133813481358136813781388139814081418142814381448145814681478148814981508151815281538154815581568157815881598160816181628163816481658166816781688169817081718172817381748175817681778178817981808181818281838184818581868187818881898190819181928193819481958196819781988199820082018202820382048205820682078208820982108211821282138214821582168217821882198220822182228223822482258226822782288229823082318232823382348235823682378238823982408241824282438244824582468247824882498250825182528253825482558256825782588259826082618262826382648265826682678268826982708271827282738274827582768277827882798280828182828283828482858286828782888289829082918292829382948295829682978298829983008301830283038304830583068307830883098310831183128313831483158316831783188319832083218322832383248325832683278328832983308331833283338334833583368337833883398340834183428343834483458346834783488349835083518352835383548355835683578358835983608361836283638364836583668367836883698370837183728373837483758376837783788379838083818382838383848385838683878388838983908391839283938394839583968397839883998400840184028403840484058406840784088409841084118412841384148415841684178418841984208421842284238424
  1. Hadoop MapReduce Change Log
  2. Release 2.8.0 - UNRELEASED
  3. INCOMPATIBLE CHANGES
  4. NEW FEATURES
  5. MAPREDUCE-6284. Add Task Attempt State API to MapReduce Application
  6. Master REST API. (Ryu Kobayashi via ozawa)
  7. MAPREDUCE-6364. Add a "Kill" link to Task Attempts page. (Ryu Kobayashi
  8. via ozawa)
  9. MAPREDUCE-6304. Specifying node labels when submitting MR jobs.
  10. (Naganarasimha G R via wangda)
  11. MAPREDUCE-6415. Create a tool to combine aggregated logs into HAR files.
  12. (Robert Kanter via kasha)
  13. IMPROVEMENTS
  14. MAPREDUCE-6471. Document distcp incremental copy
  15. (Neelesh Srinivas Salian via harsh)
  16. MAPREDUCE-5045. UtilTest#isCygwin method appears to be unused
  17. (Neelesh Srinivas Salian via harsh)
  18. MAPREDUCE-6291. Correct mapred queue usage command.
  19. (Brahma Reddu Battula via harsh)
  20. MAPREDUCE-579. Streaming "slowmatch" documentation. (harsh)
  21. MAPREDUCE-6287. Deprecated methods in org.apache.hadoop.examples.Sort
  22. (Chao Zhang via harsh)
  23. MAPREDUCE-5190. Unnecessary condition test in RandomSampler.
  24. (Jingguo Yao via harsh)
  25. MAPREDUCE-6239. Consolidate TestJobConf classes in
  26. hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core
  27. (Varun Saxena via harsh)
  28. MAPREDUCE-5807. Print usage by TeraSort job. (Rohith via harsh)
  29. MAPREDUCE-4653. TestRandomAlgorithm has an unused "import" statement.
  30. (Amir Sanjar via harsh)
  31. MAPREDUCE-6100. replace "mapreduce.job.credentials.binary" with
  32. MRJobConfig.MAPREDUCE_JOB_CREDENTIALS_BINARY for better readability.
  33. (Zhihai Xu via harsh)
  34. MAPREDUCE-6105. Inconsistent configuration in property
  35. mapreduce.reduce.shuffle.merge.percent. (Ray Chiang via harsh)
  36. MAPREDUCE-4414. Add main methods to JobConf and YarnConfiguration,
  37. for debug purposes. (Plamen Jeliazkov via harsh)
  38. MAPREDUCE-5755. MapTask.MapOutputBuffer#compare/swap should have
  39. @Override annotation. (ozawa)
  40. MAPREDUCE-6282. Reuse historyFileAbsolute.getFileSystem in
  41. CompletedJob#loadFullHistoryData for code optimization.
  42. (zxu via rkanter)
  43. MAPREDUCE-6307. Remove property
  44. mapreduce.tasktracker.taskmemorymanager.monitoringinterval.
  45. (J.Andreina via aajisaka)
  46. MAPREDUCE-6192. Create unit test to automatically compare
  47. MR related classes and mapred-default.xml (rchiang via rkanter)
  48. MAPREDUCE-6279. AM should explicity exit JVM after all services have
  49. stopped (Eric Payne via jlowe)
  50. MAPREDUCE-6079. Rename JobImpl#username to reporterUserName.
  51. (Tsuyoshi Ozawa via aajisaka)
  52. MAPREDUCE-5981. Log levels of certain MR logs can be changed to DEBUG.
  53. (Varun Saxena via devaraj)
  54. MAPREDUCE-5248. Let NNBenchWithoutMR specify the replication factor for
  55. its test (Erik Paulson via jlowe)
  56. MAPREDUCE-6174. Combine common stream code into parent class for
  57. InMemoryMapOutput and OnDiskMapOutput. (Eric Payne via gera)
  58. MAPREDUCE-6383. Pi job (QuasiMonteCarlo) should not try to read the
  59. results file if its job fails. (Harsh J via devaraj)
  60. MAPREDUCE-6354. ShuffleHandler should be able to log shuffle connections
  61. (Chang Li via jlowe)
  62. MAPREDUCE-6392. Document mapred class path options.
  63. (Brahma Reddy Battula via cnauroth)
  64. MAPREDUCE-6395. Improve the commit failure messages in MRAppMaster recovery.
  65. (Brahma Reddy Battula via gera)
  66. MAPREDUCE-6316. Task Attempt List entries should link to the task overview.
  67. (Siqi Li via gera)
  68. MAPREDUCE-6305. AM/Task log page should be able to link back to the job.
  69. (Siqi Li via gera)
  70. MAPREDUCE-6408. Queue name and user name should be printed on the job page.
  71. (Siqi Li via gera)
  72. MAPREDUCE-6384. Add the last reporting reducer info for too many fetch
  73. failure diagnostics (Chang Li via jlowe)
  74. MAPREDUCE-5762. Port MAPREDUCE-3223 and MAPREDUCE-4695 (Remove MRv1 config
  75. from mapred-default.xml) to branch-2. (aajisaka)
  76. MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. (Robert Kanter
  77. via junping_du)
  78. MAPREDUCE-6477. Replace usage of deprecated NameNode.DEFAULT_PORT in
  79. TestFileSystem. (Mingliang Liu via wheat9)
  80. MAPREDUCE-6483. Replace deprecated method NameNode.getUri() with
  81. DFSUtilClient.getNNUri() in TestMRCredentials. (Mingliang Liu via wheat9)
  82. HADOOP-12428. Fix inconsistency between log-level guards and statements.
  83. (Jagadesh Kiran N and Jackie Chang via ozawa)
  84. MAPREDUCE-6479. Add missing mapred job command options in mapreduce
  85. document. (nijel via aajisaka)
  86. MAPREDUCE-6489. Fail fast rogue tasks that write too much to local disk
  87. (Maysam Yabandeh via jlowe)
  88. OPTIMIZATIONS
  89. MAPREDUCE-6376. Add avro binary support for jhist files (Ray Chiang via
  90. jlowe)
  91. MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render()
  92. (Ray Chiang via jlowe)
  93. BUG FIXES
  94. MAPREDUCE-6314. TestPipeApplication fails on trunk.
  95. (Varun Vasudev via harsh)
  96. MAPREDUCE-5448. MapFileOutputFormat#getReaders bug with hidden
  97. files/folders. (Maysam Yabandeh via harsh)
  98. MAPREDUCE-6213. NullPointerException caused by job history server addr not
  99. resolvable. (Peng Zhang via harsh)
  100. MAPREDUCE-6281. Fix javadoc in Terasort. (Albert Chu via ozawa)
  101. MAPREDUCE-6242. Progress report log is incredibly excessive in
  102. application master. (Varun Saxena via devaraj)
  103. MAPREDUCE-6294. Remove an extra parameter described in Javadoc of
  104. TockenCache. (Brahma Reddy Battula via ozawa)
  105. MAPREDUCE-5875. Make Counter limits consistent across JobClient,
  106. MRAppMaster, and YarnChild. (Gera Shegalov via kasha)
  107. MAPREDUCE-6199. AbstractCounters are not reset completely on
  108. deserialization (adhoot via rkanter)
  109. MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which
  110. causes counter limits are not reset correctly.
  111. (Zhihai Xu via harsh)
  112. MAPREDUCE-4844. Counters / AbstractCounters have constant references not
  113. declared final. (Brahma Reddy Battula via gera)
  114. MAPREDUCE-6266. Job#getTrackingURL should consistently return a proper URL
  115. (rchiang via rkanter)
  116. MAPREDUCE-6293. Set job classloader on uber-job's LocalContainerLauncher
  117. event thread. (Sangjin Lee via gera)
  118. MAPREDUCE-6297. Task Id of the failed task in diagnostics should link to
  119. the task page. (Siqi Li via gera)
  120. MAPREDUCE-6330. Fix typo in Task Attempt API's URL in documentations.
  121. (Ryu Kobayashi via ozawa)
  122. MAPREDUCE-6333. TestEvents,TestAMWebServicesTasks,TestAppController are
  123. broken due to MAPREDUCE-6297. (Siqi Li via gera)
  124. MAPREDUCE-6341. Fix typo in mapreduce tutorial. (John Michael Luy
  125. via ozawa)
  126. MAPREDUCE-6345. Documentation fix for when CRLA is enabled for MRAppMaster
  127. logs. (Rohit Agarwal via gera)
  128. MAPREDUCE-5905. CountersStrings.toEscapedCompactStrings outputs
  129. unnecessary "null" strings. (Akira AJISAKA via ozawa)
  130. MAPREDUCE-6349. Fix typo in property org.apache.hadoop.mapreduce.
  131. lib.chain.Chain.REDUCER_INPUT_VALUE_CLASS. (Ray Chiang via ozawa)
  132. MAPREDUCE-6165. [JDK8] TestCombineFileInputFormat failed on JDK8.
  133. (Akira AJISAKA via ozawa)
  134. MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests.
  135. (Brahma Reddy Battula via aajisaka)
  136. MAPREDUCE-6342. Make POM project names consistent. (Rohith via devaraj)
  137. MAPREDUCE-3383. Duplicate job.getOutputValueGroupingComparator() in
  138. ReduceTask (Binglin Chang via jlowe)
  139. MAPREDUCE-4750. Enable NNBenchWithoutMR in MapredTestDriver (Liang Xie
  140. and Jason Lowe via raviprak)
  141. MAPREDUCE-6359. In RM HA setup, "Cluster" tab links populated with AM
  142. hostname instead of RM. (zhaoyunjiong via junping_du)
  143. MAPREDUCE-6353. Divide by zero error in MR AM when calculating available
  144. containers. (Anubhav Dhoot via kasha)
  145. MAPREDUCE-5465. Tasks are often killed before they exit on their own
  146. (Ming Ma via jlowe)
  147. MAPREDUCE-6360. TestMapreduceConfigFields is placed in wrong dir,
  148. introducing compile error (Arshad Mohammad via vinayakumarb)
  149. MAPREDUCE-6366. mapreduce.terasort.final.sync configuration in TeraSort
  150. doesn't work. (Takuya Fukudome via ozawa)
  151. MAPREDUCE-5708. Duplicate String.format in YarnOutputFiles.getSpillFileForWrite.
  152. (Konstantin Weitz via devaraj)
  153. MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists to
  154. avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state.
  155. (zhihai xu via devaraj)
  156. MAPREDUCE-6204. TestJobCounters should use new properties instead of
  157. JobConf.MAPRED_TASK_JAVA_OPTS. (Sam Liu via ozawa)
  158. MAPREDUCE-6374. Distributed Cache File visibility should check permission
  159. of full path (Chang Li via jlowe)
  160. MAPREDUCE-5965. Hadoop streaming throws error if list of input files is
  161. high. Error is: "error=7, Argument list too long at if number of input
  162. file is high" (wilfreds via rkanter)
  163. MAPREDUCE-6382. Don't escape HTML links in Diagnostics in JHS job overview.
  164. (Siqi Li via gera)
  165. MAPREDUCE-5232. Add a configuration to be able to log classpath and other
  166. system properties on mapreduce JVMs startup. (Sangjin Lee via vinodkv)
  167. MAPREDUCE-6377. JHS sorting on state column not working in webUi.
  168. (zhihai xu via devaraj)
  169. MAPREDUCE-6388. Remove deprecation warnings from JobHistoryServer classes
  170. (Ray Chiang via ozawa).
  171. MAPREDUCE-6350. JobHistory doesn't support fully-functional search.
  172. (Siqi Li via devaraj)
  173. MAPREDUCE-6389. Fix BaileyBorweinPlouffe CLI usage message. (Brahma Reddy Battula
  174. via devaraj)
  175. MAPREDUCE-6373. The logger reports total input paths but it is referring
  176. to input files. (Bibin A Chundatt via devaraj)
  177. MAPREDUCE-6405. NullPointerException in App Attempts page.
  178. (Siqi Li and Gera Shegalov via devaraj)
  179. MAPREDUCE-6403. Fix typo in the usage of NNBench.
  180. (Jagadesh Kiran N via aajisaka)
  181. MAPREDUCE-6400. Multiple shuffle transfer fails because input is closed
  182. too early (Brahma Reddy Battula, Akira AJISAKA, and Gera Shegalov via
  183. jlowe)
  184. MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host
  185. (zhihai xu via jlowe)
  186. MAPREDUCE-6121. JobResourceUpdater#compareFs() doesn't handle HA namespaces
  187. (rchiang via rkanter)
  188. MAPREDUCE-6420. Interrupted Exception in LocalContainerLauncher should be
  189. logged in warn/info level (Chang Li via jlowe)
  190. MAPREDUCE-6418. MRApp should not shutdown LogManager during shutdown
  191. (Chang Li via jlowe)
  192. MAPREDUCE-6038. A boolean may be set error in the Word Count v2.0 in
  193. MapReduce Tutorial. (Tsuyoshi Ozawa via cdouglas)
  194. MAPREDUCE-6421. Fix findbugs warning in
  195. RMContainerAllocator.reduceNodeLabelExpression.
  196. (Brahma Reddy Battula via aajisaka)
  197. MAPREDUCE-6427. Fix typo in JobHistoryEventHandler. (Ray Chiang via cdouglas)
  198. MAPREDUCE-6433. launchTime may be negative. (Zhihai Xu)
  199. MAPREDUCE-5817. Mappers get rescheduled on node transition even after all
  200. reducers are completed. (Sangjin Lee via kasha)
  201. MAPREDUCE-6357. MultipleOutputs.write() API should document that output
  202. committing is not utilized when input path is absolute.
  203. (Dustin Cote via aajisaka)
  204. MAPREDUCE-6357. MultipleOutputs.write() API should document that output
  205. committing is not utilized when input path is absolute.
  206. (Dustin Cote via aajisaka)
  207. MAPREDUCE-6452. NPE when intermediate encrypt enabled for LocalRunner.
  208. (Zhihai Xu)
  209. MAPREDUCE-5002. AM could potentially allocate a reduce container to a map
  210. attempt (Chang Li via jlowe)
  211. MAPREDUCE-6460. TestRMContainerAllocator.
  212. testAttemptNotFoundCausesRMCommunicatorException fails. (Zhihai Xu)
  213. MAPREDUCE-6484. Yarn Client uses local address instead of RM address as
  214. token renewer in a secure cluster when RM HA is enabled. (Zhihai Xu)
  215. MAPREDUCE-6480. archive-logs tool may miss applications (rkanter)
  216. MAPREDUCE-6494. Permission issue when running archive-logs tool as
  217. different users (rkanter)
  218. MAPREDUCE-6485. Create a new task attempt with failed map task priority
  219. if in-progress attempts are unassigned. (Xianyin Xin via rohithsharmaks)
  220. MAPREDUCE-6503. archive-logs tool should use HADOOP_PREFIX instead
  221. of HADOOP_HOME (rkanter)
  222. MAPREDUCE-6302. Preempt reducers after a configurable timeout irrespective
  223. of headroom. (kasha)
  224. MAPREDUCE-6495. Docs for archive-logs tool (rkanter)
  225. MAPREDUCE-6508. TestNetworkedJob fails consistently due to delegation
  226. token changes on RM. (Akira AJISAKA via junping_du)
  227. Release 2.7.2 - UNRELEASED
  228. INCOMPATIBLE CHANGES
  229. NEW FEATURES
  230. IMPROVEMENTS
  231. OPTIMIZATIONS
  232. BUG FIXES
  233. MAPREDUCE-6425. ShuffleHandler passes wrong "base" parameter to getMapOutputInfo
  234. if mapId is not in the cache. (zhihai xu via devaraj)
  235. MAPREDUCE-6426. TestShuffleHandler#testGetMapOutputInfo is failing.
  236. (zhihai xu via devaraj)
  237. MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the
  238. allocate call. (Anubhav Dhoot via kasha)
  239. MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol
  240. provider's constructor (Chang Li via ozawa)
  241. MAPREDUCE-6474. ShuffleHandler can possibly exhaust nodemanager file
  242. descriptors (Kuhu Shukla via jlowe)
  243. MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be
  244. consistent with tasks (Naganarasimha G R via jlowe)
  245. MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle
  246. multibyte record delimiters well (Vinayakumar B, Rushabh Shah, and Akira
  247. AJISAKA via jlowe)
  248. MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong
  249. position/key information for uncompressed input sometimes. (Zhihai Xu via
  250. jlowe)
  251. MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can
  252. disappear (Chang Li via jlowe)
  253. MAPREDUCE-6492. AsyncDispatcher exit with NPE on
  254. TaskAttemptImpl#sendJHStartEventForAssignedFailTask (Bibin A Chundatt via
  255. jlowe)
  256. MAPREDUCE-6518. Set SO_KEEPALIVE on shuffle connections (Chang Li via
  257. jlowe)
  258. Release 2.7.1 - 2015-07-06
  259. INCOMPATIBLE CHANGES
  260. NEW FEATURES
  261. IMPROVEMENTS
  262. OPTIMIZATIONS
  263. BUG FIXES
  264. MAPREDUCE-6252. JobHistoryServer should not fail when encountering a
  265. missing directory. (Craig Welch via devaraj)
  266. MAPREDUCE-6334. Fetcher#copyMapOutput is leaking usedMemory upon
  267. IOException during InMemoryMapOutput shuffle handler (Eric Payne via jlowe)
  268. MAPREDUCE-6339. Job history file is not flushed correctly because isTimerActive
  269. flag is not set true when flushTimerTask is scheduled. (zhihai xu via devaraj)
  270. MAPREDUCE-6259. IllegalArgumentException due to missing job submit time
  271. (zhihai xu via jlowe)
  272. MAPREDUCE-6251. Added a new config for JobClient to retry JobStatus calls so
  273. that they don't fail on history-server backed by DFSes with not so strong
  274. guarantees. (Craig Welch via vinodkv)
  275. MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at
  276. the end. (Arun Suresh via kasha)
  277. MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login)
  278. UGI to refresh log and cleaner settings. (Varun Saxena via vinodkv)
  279. Release 2.7.0 - 2015-04-20
  280. INCOMPATIBLE CHANGES
  281. NEW FEATURES
  282. MAPREDUCE-6227. DFSIO for truncate. (shv via yliu)
  283. MAPREDUCE-6228. Add truncate operation to SLive. (Plamen Jeliazkov via shv)
  284. MAPREDUCE-5583. Ability to limit running map and reduce tasks.
  285. (Jason Lowe via junping_du)
  286. IMPROVEMENTS
  287. MAPREDUCE-6149. Document override log4j.properties in MR job.
  288. (Junping Du via harsh)
  289. MAPREDUCE-6194. Bubble up final exception in failures during creation
  290. of output collectors (Varun Saxena via harsh)
  291. MAPREDUCE-5420. Remove mapreduce.task.tmp.dir from mapred-default.xml
  292. (James Carman via harsh)
  293. MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle
  294. log (Gera Shegalov via jlowe)
  295. MAPREDUCE-6046. Change the class name for logs in RMCommunicator
  296. (Sahil Takiar via devaraj)
  297. HADOOP-11032. Replace use of Guava's Stopwatch with Hadoop's StopWatch
  298. (ozawa)
  299. MAPREDUCE-6173. Document the configuration of deploying MR over distributed
  300. cache with enabling wired encryption at the same time.
  301. (Junping Du via xgong)
  302. MAPREDUCE-6141. History server leveldb recovery store (jlowe)
  303. MAPREDUCE-6150. Update document of Rumen (Masatake Iwasaki via aw)
  304. MAPREDUCE-6151. Update document of GridMix (Masatake Iwasaki via aw)
  305. MAPREDUCE-6143. add configuration for mapreduce speculative execution in
  306. MR2 (zxu via rkanter)
  307. MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors
  308. (aajisaka)
  309. MAPREDUCE-6253. Update use of Iterator to Iterable. (Ray Chiang via devaraj)
  310. MAPREDUCE-5335. Rename Job Tracker terminology in ShuffleSchedulerImpl.
  311. (devaraj via ozawa)
  312. MAPREDUCE-4431. mapred command should print the reason on killing already
  313. completed jobs. (devaraj via ozawa)
  314. MAPREDUCE-6255. Fix JobCounter's format to use grouping separator.
  315. (Ryu Kobayashi via ozawa)
  316. MAPREDUCE-6256. Removed unused private methods in o.a.h.mapreduce.Job.java.
  317. (Naganarasimha G R via ozawa)
  318. MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core.
  319. (Varun Saxena via junping_du)
  320. MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client.
  321. (Brahma Reddy Battula via aajisaka)
  322. MAPREDUCE-5612. Add javadoc for TaskCompletionEvent.Status.
  323. (Chris Palmer via aajisaka)
  324. MAPREDUCE-6248. Exposed the internal MapReduce job's information as a public
  325. API in DistCp. (Jing Zhao via vinodkv)
  326. MAPREDUCE-6263. Configurable timeout between YARNRunner terminate the
  327. application and forcefully kill. (Eric Payne via junping_du)
  328. MAPREDUCE-6265. Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable
  329. to better control to launch/kill containers. (Zhihai Xu via ozawa)
  330. MAPREDUCE-6292. Use org.junit package instead of junit.framework in
  331. TestCombineFileInputFormat. (aajisaka)
  332. OPTIMIZATIONS
  333. MAPREDUCE-6169. MergeQueue should release reference to the current item
  334. from key and value at the end of the iteration to save memory.
  335. (Zhihai Xu via kasha)
  336. MAPREDUCE-6059. Speed up history server startup time (Siqi Li via aw)
  337. MAPREDUCE-4815. Speed up FileOutputCommitter#commitJob for many output
  338. files. (Siqi Li via gera)
  339. BUG FIXES
  340. MAPREDUCE-6210. Use getApplicationAttemptId() instead of getApplicationId()
  341. for logging AttemptId in RMContainerAllocator.java (Leitao Guo via aajisaka)
  342. MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about
  343. ssl-client.xml (Yangping Wu via harsh)
  344. MAPREDUCE-5918. LineRecordReader can return the same decompressor to
  345. CodecPool multiple times (Sergey Murylev via raviprak)
  346. MAPREDUCE-6162. mapred hsadmin fails on a secure cluster (jlowe)
  347. MAPREDUCE-6049. AM JVM does not exit if MRClientService graceful shutdown
  348. fails (Rohith via devaraj)
  349. MAPREDUCE-5568. Fixed CompletedJob in JHS to show progress percentage
  350. correctly in case the number of mappers or reducers is zero. (MinJi Kim via
  351. jianhe)
  352. MAPREDUCE-6172. TestDbClasses timeouts are too aggressive (Varun Saxena
  353. via jlowe)
  354. MAPREDUCE-6160. Potential NullPointerException in MRClientProtocol
  355. interface implementation. (Rohith via jlowe)
  356. MAPREDUCE-4879. TeraOutputFormat may overwrite an existing output
  357. directory. (gera)
  358. MAPREDUCE-6045. need close the DataInputStream after open it in
  359. TestMapReduce.java (zxu via rkanter)
  360. MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems (Jens
  361. Rabe via jlowe)
  362. MAPREDUCE-3283. mapred classpath CLI does not display the complete classpath
  363. (Varun Saxena via cnauroth)
  364. MAPREDUCE-6231. Grep example job is not working on a fully-distributed
  365. cluster. (aajisaka)
  366. MAPREDUCE-6243. Fix findbugs warnings in hadoop-rumen. (Masatake Iwasaki
  367. via aajisaka)
  368. MAPREDUCE-5988. Fix dead links to the javadocs in mapreduce project.
  369. (aajisaka)
  370. MAPREDUCE-6186. Redundant call to requireJob() while displaying the conf
  371. page (Rohit Agarwal via jlowe)
  372. MAPREDUCE-6233. org.apache.hadoop.mapreduce.TestLargeSort.testLargeSort
  373. failed in trunk (zxu via rkanter)
  374. MAPREDUCE-6221. Stringifier is left unclosed in Chain#getChainElementConf().
  375. (Ted Yu via ozawa)
  376. MAPREDUCE-4286. TestClientProtocolProviderImpls passes on failure
  377. conditions. (Devaraj K via ozawa)
  378. MAPREDUCE-6261. NullPointerException if MapOutputBuffer.flush invoked
  379. twice (Tsuyoshi OZAWA via jlowe)
  380. MAPREDUCE-2815. JavaDoc does not generate correctly for
  381. MultithreadedMapRunner. (Chris Palmer via aajisaka)
  382. MAPREDUCE-6268. Fix typo in Task Attempt API's URL. (Ryu Kobayashi
  383. via ozawa)
  384. MAPREDUCE-6136. MRAppMaster doesn't shutdown file systems. (Brahma
  385. Reddy Battula via ozawa)
  386. MAPREDUCE-5657. [JDK8] Fix Javadoc errors caused by incorrect or illegal
  387. tags in doc comments. (Akira AJISAKA and Andrew Purtell via ozawa)
  388. MAPREDUCE-4742. Fix typo in nnbench#displayUsage. (Liang Xie via ozawa)
  389. MAPREDUCE-6277. Job can post multiple history files if attempt loses
  390. connection to the RM (Chang Li via jlowe)
  391. MAPREDUCE-6275. Race condition in FileOutputCommitter v2 for
  392. user-specified task output subdirs (Gera Shegalov and Siqi Li via jlowe)
  393. MAPREDUCE-6285. ClientServiceDelegate should not retry upon
  394. AuthenticationException. (Jonathan Eagles via ozawa)
  395. Release 2.6.2 - UNRELEASED
  396. INCOMPATIBLE CHANGES
  397. NEW FEATURES
  398. IMPROVEMENTS
  399. OPTIMIZATIONS
  400. BUG FIXES
  401. MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed
  402. cache files so that child processes running hadoop scripts can access these
  403. files. (Junping Du via vinodkv)
  404. MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in
  405. JobHistoryEventHandler. (Shinichi Yamashita via aajisaka)
  406. Release 2.6.1 - 2015-09-23
  407. INCOMPATIBLE CHANGES
  408. NEW FEATURES
  409. IMPROVEMENTS
  410. OPTIMIZATIONS
  411. BUG FIXES
  412. MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of
  413. reusing conections. (Kannan Rajah via ozawa)
  414. MAPREDUCE-6166. Reducers do not validate checksum of map outputs when
  415. fetching directly to disk. (Eric Payne via gera)
  416. MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken
  417. service name properly. (Jason Lowe via jianhe)
  418. MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal
  419. to a reducer. (Jason Lowe via junping_du)
  420. MAPREDUCE-6300. Task list sort by task id broken. (Siqi Li via aajisaka)
  421. MAPREDUCE-6267. Refactor JobSubmitter#copyAndConfigureFiles into it's own
  422. class. (Chris Trezzo via kasha)
  423. MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options
  424. which is a regression from MR1 (zxu via rkanter)
  425. MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM
  426. tokens when they roll-over. (Jason Lowe via vinodkv)
  427. MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge
  428. (Gera Shegalov via jlowe)
  429. MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between
  430. copySucceeded() in one thread and copyFailed() in another thread on the
  431. same host. (Junping Du via ozawa)
  432. Release 2.6.0 - 2014-11-18
  433. INCOMPATIBLE CHANGES
  434. NEW FEATURES
  435. MAPREDUCE-5910. Make MR AM resync with RM in case of work-preserving
  436. RM-restart. Contributed by Rohith
  437. MAPREDUCE-5933. Enabled MR AM to post history events to the timeline server.
  438. (Robert Kanter via zjshen)
  439. IMPROVEMENTS
  440. MAPREDUCE-5971. Move the default options for distcp -p to
  441. DistCpOptionSwitch. (clamb via wang)
  442. MAPREDUCE-5963. ShuffleHandler DB schema should be versioned with
  443. compatible/incompatible changes (Junping Du via jlowe)
  444. MAPREDUCE-6019. MapReduce changes for exposing YARN/MR endpoints on multiple
  445. interfaces. (Craig Welch, Milan Potocnik, Arpit Agarwal via xgong)
  446. MAPREDUCE-883. harchive: Document how to unarchive (Akira AJISAKA and
  447. Koji Noguchi via aw)
  448. MAPREDUCE-4791. Javadoc for KeyValueTextInputFormat should include default
  449. separator and how to change it (Akira AJISAKA via aw)
  450. MAPREDUCE-5906. Inconsistent configuration in property
  451. "mapreduce.reduce.shuffle.input.buffer.percent" (Akira AJISAKA via aw)
  452. MAPREDUCE-5974. Allow specifying multiple MapOutputCollectors with
  453. fallback. (Todd Lipcon via kasha)
  454. MAPREDUCE-5130. Add missing job config options to mapred-default.xml
  455. (Ray Chiang via Sandy Ryza)
  456. MAPREDUCE-5891. Improved shuffle error handling across NM restarts
  457. (Junping Du via jlowe)
  458. MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN
  459. scheduler resource type is memory plus cpu. (Peng Zhang and Varun Vasudev
  460. via zjshen)
  461. MAPREDUCE-6072. Remove INSTALL document (Akira AJISAKA via aw)
  462. MAPREDUCE-5970. Provide a boolean switch to enable MR-AM profiling (Gera
  463. Shegalov via jlowe)
  464. MAPREDUCE-6018. Added an MR specific config to enable emitting job history
  465. data to the timeline server. (Robert Kanter via zjshen)
  466. MAPREDUCE-6052. Supported overriding the default container-log4j.properties
  467. file per job. (Junping Du via zjshen)
  468. OPTIMIZATIONS
  469. BUG FIXES
  470. MAPREDUCE-5866. TestFixedLengthInputFormat fails in windows.
  471. (Varun Vasudev via cnauroth)
  472. MAPREDUCE-5956. Made MR AM not use maxAttempts to determine if the current
  473. attempt is the last retry. (Wangda Tan via zjshen)
  474. MAPREDUCE-5957. AM throws ClassNotFoundException with job classloader
  475. enabled if custom output format/committer is used (Sangjin Lee via jlowe)
  476. MAPREDUCE-5756. CombineFileInputFormat.getSplits() including directories
  477. in its results (Jason Dere via jlowe)
  478. MAPREDUCE-6014. New task status field in task attempts table can lead to
  479. an empty web page (Mit Desai via jlowe)
  480. MAPREDUCE-6021. MR AM should have working directory in LD_LIBRARY_PATH
  481. (jlowe)
  482. MAPREDUCE-5944. Remove MRv1 commands from CommandsManual.apt.vm
  483. (Akira AJISAKA via aw)
  484. MAPREDUCE-5943. Separate mapred commands from CommandManual.apt.vm
  485. (Akira AJISAKA via aw)
  486. MAPREDUCE-5363. Fix doc and spelling for TaskCompletionEvent#getTaskStatus
  487. and getStatus (Akira AJISAKA via aw)
  488. MAPREDUCE-6010. HistoryServerFileSystemStateStore fails to update tokens
  489. (jlowe)
  490. MAPREDUCE-5595. Typo in MergeManagerImpl.java (Akira AJISAKA via aw)
  491. MAPREDUCE-5597. Missing alternatives in javadocs for deprecated constructors
  492. in mapreduce.Job (Akira AJISAKA via aw)
  493. MAPREDUCE-5950. incorrect description in distcp2 document (Akira AJISAKA
  494. via aw)
  495. MAPREDUCE-5998. CompositeInputFormat javadoc is broken (Akira AJISAKA via
  496. aw)
  497. MAPREDUCE-5999. Fix dead link in InputFormat javadoc (Akira AJISAKA via aw)
  498. MAPREDUCE-5878. some standard JDK APIs are not part of system classes
  499. defaults (Sangjin Lee via jlowe)
  500. MAPREDUCE-6032. Made MR jobs write job history files on the default FS when
  501. the current context's FS is different. (Benjamin Zhitomirsky via zjshen)
  502. MAPREDUCE-6024. Shortened the time when Fetcher is stuck in retrying before
  503. concluding the failure by configuration. (Yunjiong Zhao via zjshen)
  504. MAPREDUCE-6036. TestJobEndNotifier fails intermittently in branch-2 (chang
  505. li via jlowe)
  506. MAPREDUCE-6012. DBInputSplit creates invalid ranges on Oracle.
  507. (Wei Yan via kasha)
  508. MAPREDUCE-6044. Fully qualified intermediate done dir path breaks per-user dir
  509. creation on Windows. (zjshen)
  510. MAPREDUCE-5885. build/test/test.mapred.spill causes release audit warnings
  511. (Chen He via jlowe)
  512. BREAKDOWN OF HDFS-6134 AND HADOOP-10150 SUBTASKS AND RELATED JIRAS
  513. MAPREDUCE-5890. Support for encrypting Intermediate
  514. data and spills in local filesystem. (asuresh via tucu)
  515. MAPREDUCE-6007. Add support to distcp to preserve raw.* namespace
  516. extended attributes. (clamb)
  517. MAPREDUCE-6041. Fix TestOptionsParser. (clamb)
  518. --
  519. MAPREDUCE-6051. Fix typos in log messages. (Ray Chiang via cdouglas)
  520. MAPREDUCE-5931. Validate SleepJob command line parameters (Gera Shegalov
  521. via jlowe)
  522. MAPREDUCE-6063. Correct spill size calculation for spills wrapping the
  523. circular buffer. (zhihai xu via cdouglas)
  524. MAPREDUCE-6071. JobImpl#makeUberDecision doesn't log that Uber mode is
  525. disabled because of too much CPUs (Tsuyoshi OZAWA via jlowe)
  526. MAPREDUCE-6075. HistoryServerFileSystemStateStore can create zero-length
  527. files (jlowe)
  528. MAPREDUCE-6070. yarn.app.am.resource.mb/cpu-vcores affects uber mode but
  529. is not documented (Tsuyoshi OZAWA via jlowe)
  530. MAPREDUCE-6090. mapred hsadmin getGroups fails to connect in some cases
  531. (Robert Kanter via jlowe)
  532. MAPREDUCE-6086. mapreduce.job.credentials.binary should allow all URIs.
  533. (Zhihai Xu via kasha)
  534. MAPREDUCE-6091. YARNRunner.getJobStatus() fails with
  535. ApplicationNotFoundException if the job rolled off the RM view (Sangjin
  536. Lee via jlowe)
  537. MAPREDUCE-6095. Enable DistributedCache for uber-mode Jobs (Gera Shegalov
  538. via jlowe)
  539. MAPREDUCE-6104. TestJobHistoryParsing.testPartialJob fails in branch-2
  540. (Mit Desai via jlowe)
  541. MAPREDUCE-6109. Fix minor typo in distcp -p usage text (Charles Lamb
  542. via aw)
  543. MAPREDUCE-6093. minor distcp doc edits (Charles Lamb via aw)
  544. MAPREDUCE-5831. Make MR client ignore unknown counters received from AM.
  545. (Junping Du via zjshen)
  546. MAPREDUCE-6073. Description of mapreduce.job.speculative.slowtaskthreshold
  547. in mapred-default should be moved into description tags (Tsuyoshi OZAWA
  548. via aw)
  549. MAPREDUCE-5796. Use current version of the archive name in
  550. DistributedCacheDeploy document (Akira AJISAKA via aw)
  551. MAPREDUCE-5945. Update the description of GenericOptionsParser -jt
  552. option (Akira AJISAKA via aw)
  553. MAPREDUCE-6087. Fixed wrong config name of
  554. MRJobConfig#MR_CLIENT_TO_AM_IPC_MAX_RETRIES_ON_TIMEOUTS. Contributed by
  555. Akira AJISAKA. (Akira AJISAKA via jianhe)
  556. MAPREDUCE-6094. TestMRCJCFileInputFormat.testAddInputPath() fails on trunk
  557. (Akira AJISAKA via jlowe)
  558. MAPREDUCE-6029. TestCommitterEventHandler fails in trunk (Mit Desai via
  559. jlowe)
  560. MAPREDUCE-6122. TestLineRecordReader may fail due to test data files checked
  561. out of git with incorrect line endings. (cnauroth)
  562. MAPREDUCE-6123. TestCombineFileInputFormat incorrectly starts 2
  563. MiniDFSCluster instances. (cnauroth)
  564. MAPREDUCE-6125. TestContainerLauncherImpl sometimes fails (Mit Desai via
  565. jlowe)
  566. MAPREDUCE-6115. TestPipeApplication#testSubmitter fails in trunk (Binglin
  567. Chang via jlowe)
  568. MAPREDUCE-5873. Shuffle bandwidth computation includes time spent waiting
  569. for maps (Siqi Li via jlowe)
  570. MAPREDUCE-5542. Killing a job just as it finishes can generate an NPE in
  571. client (Rohith via jlowe)
  572. MAPREDUCE-6126. Fixed Rumen JobBuilder to ignore NormalizedResourceEvent.
  573. (Junping Du via jianhe)
  574. MAPREDUCE-6142. Fixed test failures in TestJobHistoryEventHandler and
  575. TestMRTimelineEventHandling. (Zhijie Shen via vinodkv)
  576. MAPREDUCE-6022. map_input_file is missing from streaming job environment.
  577. (jlowe via kihwal)
  578. MAPREDUCE-6048. Fixed TestJavaSerialization failure. (Varun Vasudev via
  579. jianhe)
  580. MAPREDUCE-5960. JobSubmitter's check whether job.jar is local is incorrect
  581. with no authority in job jar path. (Gera Shegalov via jlowe)
  582. MAPREDUCE-5958. Wrong reduce task progress if map output is compressed
  583. (Emilio Coppa and jlowe via kihwal)
  584. MAPREDUCE-6156. Fetcher - connect() doesn't handle connection refused
  585. correctly (Junping Du via jlowe)
  586. Release 2.5.2 - 2014-11-19
  587. INCOMPATIBLE CHANGES
  588. NEW FEATURES
  589. IMPROVEMENTS
  590. OPTIMIZATIONS
  591. BUG FIXES
  592. Release 2.5.1 - 2014-09-05
  593. INCOMPATIBLE CHANGES
  594. NEW FEATURES
  595. IMPROVEMENTS
  596. OPTIMIZATIONS
  597. BUG FIXES
  598. MAPREDUCE-6033. Updated access check for displaying job information
  599. (Yu Gao via Eric Yang)
  600. Release 2.5.0 - 2014-08-11
  601. INCOMPATIBLE CHANGES
  602. NEW FEATURES
  603. IMPROVEMENTS
  604. MAPREDUCE-5671. NaN can be created by client and assign to Progress (Chen
  605. He via jeagles)
  606. MAPREDUCE-5665. Add audience annotations to MiniMRYarnCluster and
  607. MiniMRCluster. (Anubhav Dhoot via kasha)
  608. MAPREDUCE-5765. Update hadoop-pipes examples README (Mit Desai via jeagles)
  609. MAPREDUCE-5713. InputFormat and JobConf JavaDoc Fixes (Chen He via jeagles)
  610. MAPREDUCE-5456. TestFetcher.testCopyFromHostExtraBytes is missing (Jason
  611. Lowe via jeagles)
  612. MAPREDUCE-5804. TestMRJobsWithProfiler#testProfiler timesout (Mit Desai
  613. via kihwal)
  614. MAPREDUCE-5825. Provide diagnostics for reducers killed during ramp down
  615. (Gera Shegalov via jlowe)
  616. MAPREDUCE-5836. Fix typo in RandomTextWriter (Akira AJISAKA via jeagles)
  617. MAPREDUCE-5852. Prepare MapReduce codebase for JUnit 4.11. (cnauroth)
  618. MAPREDUCE-5639. Port DistCp2 document to trunk (Akira AJISAKA via jeagles)
  619. MAPREDUCE-5812. Make job context available to
  620. OutputCommitter.isRecoverySupported() (Mohammad Kamrul Islam via jlowe)
  621. MAPREDUCE-5638. Port Hadoop Archives document to trunk (Akira AJISAKA via
  622. jeagles)
  623. MAPREDUCE-5402. In DynamicInputFormat, change MAX_CHUNKS_TOLERABLE,
  624. MAX_CHUNKS_IDEAL, MIN_RECORDS_PER_CHUNK and SPLIT_RATIO to be configurable.
  625. (Tsuyoshi OZAWA via szetszwo)
  626. MAPREDUCE-5637. Convert Hadoop Streaming document to APT (Akira AJISAKA via
  627. jeagles)
  628. MAPREDUCE-5636. Convert MapReduce Tutorial document to APT (Akira AJISAKA
  629. via jeagles)
  630. MAPREDUCE-5774. Job overview in History UI should list reducer phases in
  631. chronological order. (Gera Shegalov via kasha)
  632. MAPREDUCE-5652. NM Recovery. ShuffleHandler should handle NM restarts.
  633. (Jason Lowe via kasha)
  634. MAPREDUCE-5861. finishedSubMaps field in LocalContainerLauncher does not
  635. need to be volatile. (Tsuyoshi OZAWA via junping_du)
  636. MAPREDUCE-5809. Enhance distcp to support preserving HDFS ACLs. (cnauroth)
  637. MAPREDUCE-5899. Support incremental data copy in DistCp. (jing9)
  638. MAPREDUCE-5886. Allow wordcount example job to accept multiple input paths.
  639. (cnauroth)
  640. MAPREDUCE-5834. Increased test-timeouts in TestGridMixClasses to avoid
  641. occassional failures. (Mit Desai via vinodkv)
  642. MAPREDUCE-5896. InputSplits should indicate which locations have the block
  643. cached in memory. (Sandy Ryza via kasha)
  644. MAPREDUCE-5844. Add a configurable delay to reducer-preemption.
  645. (Maysam Yabandeh via kasha)
  646. MAPREDUCE-5790. Made it easier to enable hprof profile options by default.
  647. (Gera Shegalov via vinodkv)
  648. OPTIMIZATIONS
  649. BUG FIXES
  650. MAPREDUCE-5759. Remove unnecessary conf load in Limits (Sandy Ryza)
  651. MAPREDUCE-5014. Extend Distcp to accept a custom CopyListing.
  652. (Srikanth Sundarrajan via amareshwari)
  653. MAPREDUCE-5775. Remove unnecessary job.setNumReduceTasks in SleepJob.createJob
  654. (jhanver chand sharma via devaraj)
  655. MAPREDUCE-4937. MR AM handles an oversized split metainfo file poorly
  656. (Eric Payne via jlowe)
  657. MAPREDUCE-5642. TestMiniMRChildTask fails on Windows.
  658. (Chuan Liu via cnauroth)
  659. MAPREDUCE-5846. Rumen doesn't understand JobQueueChangedEvent (Nathan Roberts via raviprak)
  660. MAPREDUCE-5837. MRAppMaster fails when checking on uber mode. (wheat9)
  661. MAPREDUCE-5749. TestRMContainerAllocator#testReportedAppProgress Failed
  662. (jlowe)
  663. MAPREDUCE-5884. History server uses short user name when canceling tokens
  664. (Mohammad Kamrul Islam via jlowe)
  665. MAPREDUCE-5888. Failed job leaves hung AM after it unregisters (Jason Lowe
  666. via jeagles)
  667. MAPREDUCE-5814. fat jar with *-default.xml may fail when
  668. mapreduce.job.classloader=true. (Gera Shegalov via jlowe)
  669. MAPREDUCE-5309. 2.0.4 JobHistoryParser can't parse certain failed job
  670. history files generated by 2.0.3 history server (Rushabh S Shah via jlowe)
  671. MAPREDUCE-5862. Line records longer than 2x split size aren't handled
  672. correctly (bc Wong via jlowe)
  673. MAPREDUCE-5895. Close streams properly to avoid leakage in TaskLog.
  674. (Kousuke Saruta via devaraj)
  675. MAPREDUCE-5777. Support utf-8 text with Byte Order Marker.
  676. (Zhihai Xu via kasha)
  677. MAPREDUCE-5898. distcp to support preserving HDFS extended attributes(XAttrs)
  678. (Yi Liu via umamahesh)
  679. MAPREDUCE-5920. Add Xattr option in DistCp docs. (Yi Liu via cnauroth)
  680. MAPREDUCE-5924. Changed TaskAttemptImpl to ignore TA_COMMIT_PENDING event
  681. at COMMIT_PENDING state. (Zhijie Shen via jianhe)
  682. MAPREDUCE-5939. StartTime showing up as the epoch time in JHS UI after
  683. upgrade (Chen He via jlowe)
  684. MAPREDUCE-5900. Changed to the interpret container preemption exit code as a
  685. task attempt killing event. (Mayank Bansal via zjshen)
  686. MAPREDUCE-5868. Fixed an issue with TestPipeApplication that was causing the
  687. nightly builds to fail. (Akira Ajisaka via vinodkv)
  688. MAPREDUCE-5517. Fixed MapReduce ApplicationMaster to not validate reduce side
  689. resource configuration for deciding uber-mode on map-only jobs. (Siqi Li via
  690. vinodkv)
  691. MAPREDUCE-5952. LocalContainerLauncher#renameMapOutputForReduce incorrectly
  692. assumes a single dir for mapOutIndex. (Gera Shegalov via kasha)
  693. MAPREDUCE-6002. Made MR task avoid reporting error to AM when the task process
  694. is shutting down. (Wangda Tan via zjshen)
  695. Release 2.4.1 - 2014-06-23
  696. INCOMPATIBLE CHANGES
  697. NEW FEATURES
  698. IMPROVEMENTS
  699. MAPREDUCE-5830. Added back the private API HostUtil.getTaskLogUrl(..) for
  700. binary compatibility with older clients like Hive 0.13. (Akira Ajisaka via
  701. vinodkv)
  702. OPTIMIZATIONS
  703. BUG FIXES
  704. MAPREDUCE-5714. Removed forceful JVM exit in shutDownJob.
  705. (Jinghui Wang via Eric Yang)
  706. MAPREDUCE-5818. Added "hsadmin" command into mapred.cmd. (Jian He via zjshen)
  707. MAPREDUCE-5824. Fixed test-failure of TestPipesNonJavaInputFormat in
  708. Windows. (Xuan Gong via vinodkv)
  709. MAPREDUCE-5815. Fixed test-failure of TestMRAppMaster by making MRAppMaster
  710. gracefully handle empty-queue names. (Akira Ajisaka via vinodkv)
  711. MAPREDUCE-5826. Fixed HistoryServerFileSystemStore to use right permissions
  712. on Windows for temporary files and thus also fix the test-issue with
  713. TestHistoryServerFileSystemStateStoreService. (Varun Vasudev via vinodkv)
  714. MAPREDUCE-5828. Fixed a test issue with TestMapReduceJobControl that was
  715. causing it to fail on Windows. (vinodkv)
  716. MAPREDUCE-5827. TestSpeculativeExecutionWithMRApp fails.
  717. (Zhijie Shen via cnauroth)
  718. MAPREDUCE-5833. TestRMContainerAllocator fails ocassionally.
  719. (Zhijie Shen via cnauroth)
  720. MAPREDUCE-5832. Fixed TestJobClient to not fail on JDK7 or on Windows. (Jian
  721. He and Vinod Kumar Vavilapalli via vinodkv)
  722. MAPREDUCE-5841. uber job doesn't terminate on getting mapred job kill
  723. (Sangjin Lee via jlowe)
  724. MAPREDUCE-5843. Fixed TestMRKeyValueTextInputFormat to not leak files and
  725. thus avoid failing on Windows. (Varun Vasudev via vinodkv)
  726. MAPREDUCE-5835. Killing Task might cause the job to go to ERROR state
  727. (Ming Ma via jlowe)
  728. MAPREDUCE-5821. Avoid unintentional reallocation of byte arrays in segments
  729. during merge. (Todd Lipcon via cdouglas)
  730. Release 2.4.0 - 2014-04-07
  731. INCOMPATIBLE CHANGES
  732. NEW FEATURES
  733. MAPREDUCE-5787. Added the ability to keep alive shuffle connections in the
  734. MapReduce shuffle-handler. (Rajesh Balamohan via vinodkv)
  735. IMPROVEMENTS
  736. MAPREDUCE-5464. Add analogs of the SLOTS_MILLIS counters that jive with the
  737. YARN resource model (Sandy Ryza)
  738. MAPREDUCE-5732. Report proper queue when job has been automatically placed
  739. (Sandy Ryza)
  740. MAPREDUCE-5699. Allow setting tags on MR jobs (kasha)
  741. MAPREDUCE-5761. Added a simple log message to denote when encrypted shuffle
  742. is on in the shuffle-handler. (Jian He via vinodkv)
  743. MAPREDUCE-5754. Preserve Job diagnostics in history (Gera Shegalov via
  744. jlowe)
  745. MAPREDUCE-5773. Provide dedicated MRAppMaster syslog length limit (Gera
  746. Shegalov via jlowe)
  747. MAPREDUCE-5553. Allow users to easily access
  748. completed/pending/successful/failed tasks on MR AM web-ui. (Paul Han via
  749. acmurthy)
  750. MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to
  751. handle cross platform application submissions. (Jian He via vinodkv)
  752. MAPREDUCE-2349. Modified FileInputFormat to be able to issue file and block
  753. location calls in parallel. (Siddharth Seth via vinodkv)
  754. OPTIMIZATIONS
  755. BUG FIXES
  756. MAPREDUCE-5746. Job diagnostics can implicate wrong task for a failed job.
  757. (Jason Lowe via kasha)
  758. MAPREDUCE-5670. CombineFileRecordReader should report progress when moving
  759. to the next file (Chen He via jlowe)
  760. MAPREDUCE-5757. ConcurrentModificationException in JobControl.toList
  761. (jlowe)
  762. MAPREDUCE-5770. Fixed MapReduce ApplicationMaster to correctly redirect
  763. to the YARN's web-app proxy with the correct scheme prefix. (Jian He via
  764. vinodkv)
  765. MAPREDUCE-5768. TestMRJobs.testContainerRollingLog fails on trunk (Gera
  766. Shegalov via jlowe)
  767. MAPREDUCE-5780. SliveTest should use the specified path to get the
  768. particular FileSystem instead of using the default FileSystem. (szetszwo)
  769. MAPREDUCE-5028. Fixed a bug in MapTask that was causing mappers to fail
  770. when a large value of io.sort.mb is set. (Karthik Kambatla via vinodkv)
  771. MAPREDUCE-5778. JobSummary does not escape newlines in the job name (Akira
  772. AJISAKA via jlowe)
  773. MAPREDUCE-5789. Average Reduce time is incorrect on Job Overview page
  774. (Rushabh S Shah via jlowe)
  775. MAPREDUCE-5794. SliveMapper always uses default FileSystem. (szetszwo via
  776. Arpit Agarwal)
  777. MAPREDUCE-5751. MR app master fails to start in some cases if
  778. mapreduce.job.classloader is true (Sangjin Lee via jlowe)
  779. MAPREDUCE-5688. TestStagingCleanup fails intermittently with JDK7 (Mit
  780. Desai via jeagles)
  781. MAPREDUCE-5769. Unregistration to RM should not be called if AM is crashed
  782. before registering with RM (Rohith via jlowe)
  783. MAPREDUCE-5570. Map task attempt with fetch failure has incorrect attempt
  784. finish time (Rushabh S Shah via jlowe)
  785. MAPREDUCE-5806. Fixed a bug in MRAppMaster so as to enable users to properly
  786. override HADOOP_ROOT_LOGGER or HADOOP_CLIENT_OPTS. (Varun Vasudev via
  787. vinodkv)
  788. MAPREDUCE-5791. Shuffle phase is slow in Windows -
  789. FadviseFileRegion::transferTo does not read disks efficiently.
  790. (Nikola Vujic via cnauroth)
  791. MAPREDUCE-5795. Fixed MRAppMaster to record the correct job-state after it
  792. recovers from a commit during a previous attempt. (Xuan Gong via vinodkv)
  793. MAPREDUCE-5805. Fixed MapReduce JobHistory encoding of queue-name to escape
  794. hyphens and thus avoid parsing errors. (Akira AJISAKA via vinodkv)
  795. MAPREDUCE-5810. Removed the faulty and failing streaming test
  796. TestStreamingTaskLog. (Akira Ajisaka via vinodkv)
  797. MAPREDUCE-5813. Fix YarnChild to explicitly load job.xml from the
  798. local-filesystem, rather than rely on the classpath. (Gera Shegalov via
  799. acmurthy)
  800. Release 2.3.1 - UNRELEASED
  801. INCOMPATIBLE CHANGES
  802. NEW FEATURES
  803. IMPROVEMENTS
  804. OPTIMIZATIONS
  805. BUG FIXES
  806. Release 2.3.0 - 2014-02-18
  807. INCOMPATIBLE CHANGES
  808. NEW FEATURES
  809. MAPREDUCE-5265. History server admin service to refresh user and superuser
  810. group mappings (Ashwin Shankar via jlowe)
  811. MAPREDUCE-5356. Ability to refresh aggregated log retention period and
  812. check interval (Ashwin Shankar via jlowe)
  813. MAPREDUCE-5386. Ability to refresh history server job retention and job
  814. cleaner settings (Ashwin Shankar via jlowe)
  815. MAPREDUCE-5411. Refresh size of loaded job cache on history server (Ashwin
  816. Shankar via jlowe)
  817. MAPREDUCE-5332. Support token-preserving restart of history server (jlowe)
  818. IMPROVEMENTS
  819. MAPREDUCE-5329. Allow MR applications to use additional AuxServices,
  820. which are compatible with the default MapReduce shuffle.
  821. (Avner BenHanoch via sseth)
  822. MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters (Tzuyoshi Ozawa via Sandy
  823. Ryza)
  824. MAPREDUCE-5457. Add a KeyOnlyTextOutputReader to enable streaming to write
  825. out text files without separators (Sandy Ryza)
  826. MAPREDUCE-5596. Allow configuring the number of threads used to serve
  827. shuffle connections (Sandy Ryza via jlowe)
  828. MAPREDUCE-434. LocalJobRunner limited to single reducer (Sandy Ryza and
  829. Aaron Kimball via Sandy Ryza)
  830. MAPREDUCE-4421. Run MapReduce framework via the distributed cache (jlowe)
  831. MAPREDUCE-1176. FixedLengthInputFormat and FixedLengthRecordReader
  832. (Mariappan Asokan and BitsOfInfo via Sandy Ryza)
  833. MAPREDUCE-5613. DefaultSpeculator holds and checks hashmap that is always
  834. empty (Gera Shegalov via Sandy Ryza)
  835. MAPREDUCE-5431. Missing pom dependency in MR-client (Timothy St. Clair
  836. via stevel)
  837. MAPREDUCE-5624 Move grizzly-test and junit dependencies to test scope
  838. (Ted Yu via stevel)
  839. MAPREDUCE-5481. Enable uber jobs to have multiple reducers (Sandy Ryza)
  840. MAPREDUCE-5052. Job History UI and web services confusing job start time and
  841. job submit time (Chen He via jeagles)
  842. MAPREDUCE-5692. Add explicit diagnostics when a task attempt is killed due
  843. to speculative execution (Gera Shegalov via Sandy Ryza)
  844. MAPREDUCE-5550. Task Status message (reporter.setStatus) not shown in UI
  845. with Hadoop 2.0 (Gera Shegalov via Sandy Ryza)
  846. MAPREDUCE-3310. Custom grouping comparator cannot be set for Combiners (tucu)
  847. MAPREDUCE-5672. Provide optional RollingFileAppender for container log4j
  848. (syslog) (Gera Shegalov via jlowe)
  849. MAPREDUCE-5725. Make explicit that TestNetworkedJob relies on the Capacity
  850. Scheduler (Sandy Ryza)
  851. MAPREDUCE-5744. Job hangs because
  852. RMContainerAllocator$AssignedRequests.preemptReduce() violates the
  853. comparator contract (Gera Shegalov via kasha)
  854. OPTIMIZATIONS
  855. MAPREDUCE-4680. Job history cleaner should only check timestamps of files in
  856. old enough directories (Robert Kanter via Sandy Ryza)
  857. MAPREDUCE-5484. YarnChild unnecessarily loads job conf twice (Sandy Ryza)
  858. MAPREDUCE-5487. In task processes, JobConf is unnecessarily loaded again
  859. in Limits (Sandy Ryza)
  860. MAPREDUCE-5601. ShuffleHandler fadvises file regions as DONTNEED even when
  861. fetch fails (Sandy Ryza)
  862. BUG FIXES
  863. MAPREDUCE-5569. FloatSplitter is not generating correct splits (Nathan
  864. Roberts via jlowe)
  865. MAPREDUCE-5546. mapred.cmd on Windows set HADOOP_OPTS incorrectly (Chuan Liu
  866. via cnauroth)
  867. MAPREDUCE-5518. Fixed typo "can't read paritions file". (Albert Chu
  868. via devaraj)
  869. MAPREDUCE-5561. org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl
  870. testcase failing on trunk (Karthik Kambatla via jlowe)
  871. MAPREDUCE-5598. TestUserDefinedCounters.testMapReduceJob is flakey
  872. (Robert Kanter via jlowe)
  873. MAPREDUCE-5604. TestMRAMWithNonNormalizedCapabilities fails on Windows due to
  874. exceeding max path length. (cnauroth)
  875. MAPREDUCE-5451. MR uses LD_LIBRARY_PATH which doesn't mean anything in
  876. Windows. (Yingda Chen via cnauroth)
  877. MAPREDUCE-5409. MRAppMaster throws InvalidStateTransitonException: Invalid
  878. event: TA_TOO_MANY_FETCH_FAILURE at KILLED for TaskAttemptImpl (Gera
  879. Shegalov via jlowe)
  880. MAPREDUCE-5674. Missing start and finish time in mapred.JobStatus.
  881. (Chuan Liu via cnauroth)
  882. MAPREDUCE-5650. Job fails when hprof mapreduce.task.profile.map/reduce.params
  883. is specified (Gera Shegalov via Sandy Ryza)
  884. MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal
  885. task-state (Ashwin Shankar via jlowe)
  886. MAPREDUCE-5380. Invalid mapred command should return non-zero exit code
  887. (Stephen Chu via jlowe)
  888. MAPREDUCE-5404. HSAdminServer does not use ephemeral ports in minicluster
  889. mode (Ted Yu via jlowe)
  890. MAPREDUCE-5522. Incorrect oreder expected from JobQueueInfo (Jinghui Wang
  891. via bobby)
  892. MAPREDUCE-5514. Fix TestRMContainerAllocator. (Zhijie Shen via acmurthy)
  893. MAPREDUCE-5102. fix coverage org.apache.hadoop.mapreduce.lib.db and
  894. org.apache.hadoop.mapred.lib.db (Aleksey Gorshkov, Andrey Klochkov, and
  895. Nathan Roberts via jlowe)
  896. MAPREDUCE-5585. TestCopyCommitter#testNoCommitAction Fails on JDK7
  897. (jeagles)
  898. MAPREDUCE-5186. mapreduce.job.max.split.locations causes some splits
  899. created by CombineFileInputFormat to fail (Robert Parker and Jason Lowe
  900. via jlowe)
  901. MAPREDUCE-5610. TestSleepJob fails in jdk7 (Jonathan Eagles via jlowe)
  902. MAPREDUCE-5616. MR Client-AppMaster RPC max retries on socket timeout is too
  903. high. (cnauroth)
  904. MAPREDUCE-5625. TestFixedLengthInputFormat fails in jdk7 environment
  905. (Mariappan Asokan via jeagles)
  906. MAPREDUCE-5631. TestJobEndNotifier.testNotifyRetries fails with Should
  907. have taken more than 5 seconds in jdk7 (Jonathan Eagles via jlowe)
  908. MAPREDUCE-5645. TestFixedLengthInputFormat fails with native libs (Mit
  909. Desai via jeagles)
  910. MAPREDUCE-5632. TestRMContainerAllocator#testUpdatedNodes fails (jeagles)
  911. MAPREDUCE-5656. bzip2 codec can drop records when reading data in splits
  912. (jlowe)
  913. MAPREDUCE-5623. TestJobCleanup fails because of RejectedExecutionException
  914. and NPE. (jlowe)
  915. MAPREDUCE-5679. TestJobHistoryParsing has race condition (Liyin Liang via
  916. jlowe)
  917. MAPREDUCE-5687. Fixed failure in TestYARNRunner caused by YARN-1446. (Jian He
  918. via vinodkv)
  919. MAPREDUCE-5694. Fixed MR AppMaster to shutdown the LogManager so as to avoid
  920. losing syslog in some conditions. (Mohammad Kamrul Islam via vinodkv)
  921. MAPREDUCE-5685. Fixed a bug with JobContext getCacheFiles API inside the
  922. WrappedReducer class. (Yi Song via vinodkv)
  923. MAPREDUCE-5689. MRAppMaster does not preempt reducers when scheduled maps
  924. cannot be fulfilled. (lohit via kasha)
  925. MAPREDUCE-5724. JobHistoryServer does not start if HDFS is not running.
  926. (tucu)
  927. MAPREDUCE-5729. mapred job -list throws NPE (kasha)
  928. MAPREDUCE-5693. Restore MRv1 behavior for log flush (Gera Shegalov via
  929. jlowe)
  930. MAPREDUCE-5723. MR AM container log can be truncated or empty.
  931. (Mohammad Kamrul Islam via kasha)
  932. MAPREDUCE-5743. Fixed the test failure in TestRMContainerAllocator.
  933. (Ted Yu and Vinod Kumar Vavilapalli via zjshen)
  934. Release 2.2.0 - 2013-10-13
  935. INCOMPATIBLE CHANGES
  936. NEW FEATURES
  937. IMPROVEMENTS
  938. OPTIMIZATIONS
  939. BUG FIXES
  940. MAPREDUCE-5504. mapred queue -info inconsistent with types (Kousuke Saruta
  941. via tgraves)
  942. MAPREDUCE-5488. Changed MR client to keep trying to reach the application
  943. when it sees that on attempt's AM is down. (Jian He via vinodkv)
  944. MAPREDUCE-5515. Fixed MR AM's webapp to depend on a new config
  945. mapreduce.ssl.enabled to enable https and disabling it by default as MR AM
  946. needs to set up its own certificates etc and not depend on clusters'.
  947. (Omkar Vinit Joshi via vinodkv)
  948. MAPREDUCE-5505. Clients should be notified job finished only after job
  949. successfully unregistered (Zhijie Shen via bikas)
  950. MAPREDUCE-5503. Fixed a test issue in TestMRJobClient. (Jian He via vinodkv)
  951. MAPREDUCE-5170. Fixed a wrong log message in CombineFileInputFormat class.
  952. (Sangjin Lee via vinodkv)
  953. MAPREDUCE-5525. Increase timeout of TestDFSIO.testAppend and
  954. TestMRJobsWithHistoryService.testJobHistoryData. (Chuan Liu via cnauroth)
  955. MAPREDUCE-5513. ConcurrentModificationException in JobControl (Robert
  956. Parker via jlowe)
  957. MAPREDUCE-5531. Fix compat with hadoop-1 in mapreduce.(TaskID,
  958. TaskAttemptID) by re-introducing missing constructors. (Robert Kanter via
  959. acmurthy)
  960. MAPREDUCE-5545. org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.testCommitWindow
  961. times out (Robert Kanter via jlowe)
  962. MAPREDUCE-5529. Fix compat with hadoop-1 in mapred.TotalOrderPartitioner
  963. by re-introducing (get,set)PartitionFile which takes in JobConf. (Robert
  964. Kanter via acmurthy)
  965. MAPREDUCE-5538. Fixed MR AppMaster to send job-notification URL only after
  966. the job is really done - a bug caused by MAPREDUCE-5505. (Zhijie Shen via
  967. vinodkv)
  968. MAPREDUCE-5551. Fix compat with hadoop-1 in
  969. SequenceFileAsBinaryOutputFormat.WritableValueBytes by re-introducing
  970. missing constructors. (Zhijie Shen via acmurthy)
  971. MAPREDUCE-5544. JobClient#getJob loads job conf twice. (Sandy Ryza)
  972. MAPREDUCE-5536. Fixed MR AM and JHS to respect
  973. mapreduce.jobhistory.webapp.https.address. (Omkar Vinit Joshi via vinodkv)
  974. MAPREDUCE-5530. Fix compat with hadoop-1 in
  975. mapred.lib.CombinFileInputFormat by re-introducing
  976. isSplittable(FileSystem, Path) api and ensuring semantic compatibility.
  977. (Robert Kanter via acmurthy)
  978. MAPREDUCE-5459. Update documentation on how to run MRv1 examples on YARN.
  979. (Zhijie Shen via acmurthy)
  980. MAPREDUCE-5554. hdfs-site.xml included in hadoop-mapreduce-client-jobclient
  981. tests jar is breaking tests for downstream components (Robert Kanter via
  982. Sandy Ryza)
  983. MAPREDUCE-5489. MR jobs hangs as it does not use the node-blacklisting
  984. feature in RM requests (Zhijie Shen via bikas)
  985. MAPREDUCE-5442. $HADOOP_MAPRED_HOME/$HADOOP_CONF_DIR setting not working on
  986. Windows. (Yingda Chen via cnauroth)
  987. MAPREDUCE-5533. Fixed MR speculation code to track any TaskAttempts that
  988. aren't heart-beating for a while, so that we can aggressively speculate
  989. instead of waiting for task-timeout (Xuan Gong via vinodkv)
  990. MAPREDUCE-5562. Fixed MR App Master to perform pending tasks like staging-dir
  991. cleanup, sending job-end notification correctly when unregister with RM
  992. fails. (Zhijie Shen via vinodkv)
  993. Release 2.1.1-beta - 2013-09-23
  994. INCOMPATIBLE CHANGES
  995. NEW FEATURES
  996. IMPROVEMENTS
  997. MAPREDUCE-5478. TeraInputFormat unnecessarily defines its own FileSplit
  998. subclass (Sandy Ryza)
  999. MAPREDUCE-5497. Changed MRAppMaster to sleep only after doing everything else
  1000. but just before ClientService to avoid race conditions during RM restart.
  1001. (Jian He via vinodkv)
  1002. MAPREDUCE-5379. Include token tracking ids in jobconf. (kkambatl via tucu)
  1003. MAPREDUCE-5523. Added separate configuration properties for https for JHS
  1004. without which even when https is enabled, it starts on http port itself.
  1005. (Omkar Vinit Joshi via vinodkv)
  1006. OPTIMIZATIONS
  1007. MAPREDUCE-5446. TestJobHistoryEvents and TestJobHistoryParsing have race
  1008. conditions (jlowe via kihwal)
  1009. MAPREDUCE-5462. In map-side sort, swap entire meta entries instead of
  1010. indexes for better cache performance. (Sandy Ryza)
  1011. MAPREDUCE-1981. Improve getSplits performance by using listLocatedStatus
  1012. (Hairong Kuang and Jason Lowe via jlowe)
  1013. BUG FIXES
  1014. MAPREDUCE-5385. Fixed a bug with JobContext getCacheFiles API. (Omkar Vinit
  1015. Joshi via vinodkv)
  1016. MAPREDUCE-5428. HistoryFileManager doesn't stop threads when service is
  1017. stopped (Karthik Kambatla via jlowe)
  1018. MAPREDUCE-5251. Reducer should not implicate map attempt if it has
  1019. insufficient space to fetch map output (Ashwin Shankar via jlowe)
  1020. MAPREDUCE-5317. Stale files left behind for failed jobs (Ravi Prakash via
  1021. jlowe)
  1022. MAPREDUCE-5358. MRAppMaster throws invalid transitions for JobImpl
  1023. (Devaraj K via jlowe)
  1024. MAPREDUCE-3193. FileInputFormat doesn't read files recursively in the
  1025. input path dir (Devaraj K via jlowe)
  1026. MAPREDUCE-5440. TestCopyCommitter Fails on JDK7 (Robert Parker via jlowe)
  1027. MAPREDUCE-5367. Local jobs all use same local working directory
  1028. (Sandy Ryza)
  1029. MAPREDUCE-5425. Junit in TestJobHistoryServer failing in jdk 7 (Robert
  1030. Parker via jlowe)
  1031. MAPREDUCE-5454. TestDFSIO fails intermittently on JDK7 (Karthik Kambatla
  1032. via Sandy Ryza)
  1033. MAPREDUCE-5001. LocalJobRunner has race condition resulting in job
  1034. failures (Sandy Ryza via jlowe)
  1035. MAPREDUCE-5466. Changed MR AM to not promote history files of intermediate
  1036. AMs in case they are exiting because of errors and thus help history-server
  1037. pick up the right history file for the last successful AM. (Jian He via
  1038. vinodkv)
  1039. MAPREDUCE-5468. Fix MR AM recovery for map-only jobs. (vinodkv via
  1040. acmurthy)
  1041. MAPREDUCE-5470. LocalJobRunner does not work on Windows. (Sandy Ryza via
  1042. cnauroth)
  1043. MAPREDUCE-5476. Changed MR AM recovery code to cleanup staging-directory
  1044. only after unregistering from the RM. (Jian He via vinodkv)
  1045. MAPREDUCE-5483. revert MAPREDUCE-5357. (rkanter via tucu)
  1046. MAPREDUCE-5441. Changed MR AM to return RUNNING state if exiting when RM
  1047. commands to reboot, so that client can continue to track the overall job.
  1048. (Jian He via vinodkv)
  1049. MAPREDUCE-5475. MRClientService does not verify ACLs properly (jlowe)
  1050. MAPREDUCE-5414. TestTaskAttempt fails in JDK7 with NPE (Nemon Lou via
  1051. devaraj)
  1052. MAPREDUCE-5020. Compile failure with JDK8 (Trevor Robinson via tgraves)
  1053. MAPREDUCE-5164. mapred job and queue commands omit HADOOP_CLIENT_OPTS
  1054. (Nemon Lou via devaraj)
  1055. MAPREDUCE-5493. Cleanup in-memory & on-disk segments to prevent leak on
  1056. shuffle completion. (jlowe via acmurthy)
  1057. Release 2.1.0-beta - 2013-08-22
  1058. INCOMPATIBLE CHANGES
  1059. MAPREDUCE-4067. Changed MRClientProtocol api to throw IOException only (Xuan
  1060. Gong via vinodkv)
  1061. MAPREDUCE-5234. Change mapred.TaskReport and mapreduce.TaskReport for binary
  1062. compatibility with mapred in 1.x but incompatible with 0.23.x. (Mayank Bansal
  1063. via vinodkv)
  1064. MAPREDUCE-5156. Change hadoop examples ProgramDriver to be able to run
  1065. 1.x examples jar on top of YARN. This change breaks 0.23.x direct usages of
  1066. ProgramDriver. (Zhijie Shen via vinodkv)
  1067. MAPREDUCE-5233. Add methods that are changed or removed from JobControl.Job
  1068. when compared to 1.x. This breaks 0.23.x users of one API in Job. (Mayank
  1069. Bansal via vinodkv)
  1070. MAPREDUCE-5237. Add methods that were removed from ClusterStatus back into
  1071. 2.x so as to be compatible with 1.x. Incompatible as
  1072. ClusterStatus.UNINITIALIZED_MEMORY_VALUE is a long now and so breaks 0.23.x
  1073. but it shouldn't be a big deal in reality. (Zhijie Shen via vinodkv)
  1074. NEW FEATURES
  1075. HADOOP-8562. Enhancements to support Hadoop on Windows Server and Windows
  1076. Azure environments. (See breakdown of tasks below for subtasks and
  1077. contributors)
  1078. IMPROVEMENTS
  1079. MAPREDUCE-3008. Improvements to cumulative CPU emulation for short running
  1080. tasks in Gridmix. (amarrk via tgraves)
  1081. MAPREDUCE-5033. mapred shell script should respect usage flags
  1082. (--help -help -h). (Andrew Wang via atm)
  1083. MAPREDUCE-4892. Modify CombineFileInputFormat to not skew input slits'
  1084. allocation on small clusters. (Bikas Saha via vinodkv)
  1085. MAPREDUCE-4990. Construct debug strings conditionally in
  1086. ShuffleHandler.Shuffle#sendMapOutput(). (kkambatl via tucu)
  1087. MAPREDUCE-4875. coverage fixing for org.apache.hadoop.mapred
  1088. (Aleksey Gorshkov via bobby)
  1089. MAPREDUCE-5129. Allow tags to JobHistory for deeper analytics. (billie via
  1090. acmurthy)
  1091. MAPREDUCE-3787. [Gridmix] Optimize job monitoring and STRESS mode for
  1092. faster job submission. (amarrk via tgraves)
  1093. MAPREDUCE-5079. Changes job recovery to restore state directly from job
  1094. history, instaed of simulating state machine events.
  1095. (Jason Lowe and Robert Parker via sseth)
  1096. MAPREDUCE-4981. Add WordMean, WordMedian, WordStandardDeviation
  1097. to ExamplesDriver. (Plamen Jeliazkov via shv)
  1098. MAPREUDUCE-5059. Change average merge time on Job overview page to be the
  1099. time delta between the end of the shuffle and the start of the reduce.
  1100. (Omkar Vinit Joshi via vinodkv)
  1101. MAPREDUCE-4985. Add compression option to TestDFSIO usage.
  1102. (Plamen Jeliazkov via shv)
  1103. MAPREDUCE-5152. Make MR App to simply pass through the container from RM
  1104. instead of extracting and populating information itself to start any
  1105. container. (vinodkv)
  1106. MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs
  1107. anyways after YARN-561. (Xuan Gong via vinodkv)
  1108. MAPREDUCE-5069. add concrete common implementations of
  1109. CombineFileInputFormat (Sangjin Lee via bobby)
  1110. MAPREDUCE-5145. Changed default max-attempts to be more than one for MR jobs
  1111. inline with YARN. (Zhijie Shen via vinodkv)
  1112. MAPREDUCE-5036. Default shuffle handler port should not be 8080.
  1113. (Sandy Ryza via tomwhite)
  1114. MAPREDUCE-5159. Change ValueAggregatorJob to add APIs which can support
  1115. binary compatibility with hadoop-1 examples. (Zhijie Shen via vinodkv)
  1116. MAPREDUCE-5157. Bring back old sampler related code so that we can support
  1117. binary compatibility with hadoop-1 sorter example. (Zhijie Shen via vinodkv)
  1118. MAPREDUCE-5222. Bring back some methods and constants in Jobclient for
  1119. binary compatibility with mapred in 1.x. (Karthik Kambatla via vinodkv)
  1120. MAPREDUCE-5235. Bring back old fields and exceptions in Counters for
  1121. binary compatibility with mapred in 1.x. (Mayank Bansal via vinodkv)
  1122. MAPREDUCE-5246. Specify application-type at the time of job submission after
  1123. YARN-563. (Mayank Bansal via vinodkv)
  1124. MAPREDUCE-5230. Bring back NLineInputFormat.createFileSplit for binary
  1125. compatibility with mapred in 1.x (Mayank Bansal via vinodkv)
  1126. MAPREDUCE-5270. Migrated MR app from using BuilderUtil factory methods to
  1127. individual record factory methods. (Jian He via vinodkv)
  1128. MAPREDUCE-5263. Bring back old methods and fields in
  1129. filecache.DistributedCache for binary compatibility with mapred in 1.x.
  1130. (Zhijie Shen via vinodkv)
  1131. MAPREDUCE-5228. Bring back FileInputFormat.Counter and
  1132. FileOuputFormat.Counter for binary compatibility with 1.x mapred APIs.
  1133. (Mayank Bansal via vinodkv)
  1134. MAPREDUCE-5176. Add annotation for tagging tasks as responsive to
  1135. preemption. (Carlo Curino, cdouglas)
  1136. MAPREDUCE-5275. Bring back a couple of APIs in mapreduce.security.TokenCache
  1137. for binary compatibility with 1.x mapreduce APIs. (Mayank Bansal via vinodkv)
  1138. MAPREDUCE-5231. Bring back a constructor in mapred's
  1139. DBInputFormat.DBRecordReader for binary compatibility with 1.x mapred APIs.
  1140. (Zhijie Shen via vinodkv)
  1141. MAPREDUCE-5273. Bring back a couple of protected variables in mapred and
  1142. mapreduce CombineFileRecordReader for binary compatibility with 1.x APIs.
  1143. (Mayank Bansal via vinodkv)
  1144. MAPREDUCE-5280. Bring back removed constructor and a method in mapreduce
  1145. ClusterMetrics for binary compatibility with 1.x APIs. (Mayank Bansal via
  1146. vinodkv)
  1147. MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. (Jian He
  1148. via vinodkv)
  1149. MAPREDUCE-5229. Put back FileOutputCommitter.TEMP_DIR_NAME in mapreduce for
  1150. binary compatibility with 1.x APIs. (Zhijie Shen via vinodkv)
  1151. MAPREDUCE-5274. Bring back SecureShuffleUtils.toHex in mapreduce for binary
  1152. compatibility with 1.x APIs. (Mayank Bansal via vinodkv)
  1153. MAPREDUCE-5300. Fix backward incompatibility for
  1154. o.a.h.mapreduce.filecache.DistributedCache. (Zhijie Shen via acmurthy)
  1155. MAPREDUCE-5283. Over 10 different tests have near identical
  1156. implementations of AppContext (Sandy Ryza via jlowe)
  1157. MAPREDUCE-5199. Removing ApplicationTokens file as it is no longer needed.
  1158. (Daryn Sharp via vinodkv)
  1159. MAPREDUCE-5192. Allow for alternate resolutions of TaskCompletionEvents.
  1160. (cdouglas via acmurthy)
  1161. MAPREDUCE-5184. Document compatibility for MapReduce applications in
  1162. hadoop-2 vis-a-vis hadoop-1. (Zhijie Shen via acmurthy)
  1163. MAPREDUCE-5194. Heed interrupts during Fetcher shutdown. (cdouglas)
  1164. MAPREDUCE-5326. Added version to shuffle header. (Zhijie Shen via
  1165. acmurthy)
  1166. MAPREDUCE-5333. Add test that verifies MRAM works correctly when sending
  1167. requests with non-normalized capabilities. (ywskycn via tucu)
  1168. MAPREDUCE-5398. MR changes for YARN-513 (Jian He via bikas)
  1169. OPTIMIZATIONS
  1170. MAPREDUCE-4974. Optimising the LineRecordReader initialize() method
  1171. (Gelesh via bobby)
  1172. MAPREDUCE-5268. Improve history server startup performance (Karthik
  1173. Kambatla via jlowe)
  1174. MAPREDUCE-5352. Optimize node local splits generated by
  1175. CombineFileInputFormat. (sseth)
  1176. BUG FIXES
  1177. MAPREDUCE-4671. AM does not tell the RM about container requests which are
  1178. no longer needed. (Bikas Saha via sseth)
  1179. MAPREDUCE-4994. -jt generic command line option does not work. (sandyr via tucu)
  1180. MAPREDUCE-5000. Fixes getCounters when speculating by fixing the selection
  1181. of the best attempt for a task. (Jason Lowe via sseth)
  1182. MAPREDUCE-4994. Addendum fixing testcases failures. (sandyr via tucu)
  1183. MAPREDUCE-4846. Some JobQueueInfo methods are public in MR1 but protected
  1184. in MR2. (Sandy Ryza via tomwhite)
  1185. MAPREDUCE-5013. mapred.JobStatus compatibility: MR2 missing constructors
  1186. from MR1. (Sandy Ryza via tomwhite)
  1187. MAPREDUCE-4951. Container preemption interpreted as task failure.
  1188. (Sandy Ryza via tomwhite)
  1189. MAPREDUCE-5008. Merger progress miscounts with respect to EOF_MARKER.
  1190. (Sandy Ryza via tomwhite)
  1191. MAPREDUCE-4693. History server should include counters for failed tasks.
  1192. (Xuan Gong via sseth)
  1193. MAPREDUCE-4896. mapred queue -info spits out ugly exception when queue does
  1194. not exist. (sandyr via tucu)
  1195. MAPREDUCE-3685. Fix bugs in MergeManager to ensure compression codec is
  1196. appropriately used and that on-disk segments are correctly sorted on
  1197. file-size. (Anty Rao and Ravi Prakash via acmurthy)
  1198. MAPREDUCE-4571. TestHsWebServicesJobs fails on jdk7. (tgraves via tucu)
  1199. MAPREDUCE-4716. TestHsWebServicesJobsQuery.testJobsQueryStateInvalid
  1200. fails with jdk7. (tgraves via tucu)
  1201. MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream
  1202. does not close the wrapped InputStream. (Chris Nauroth via szetszwo)
  1203. MAPREDUCE-3872. Fix an event handling races in ContainerLauncherImpl.
  1204. (Robert Kanter via sseth)
  1205. MAPREDUCE-5062. Fix MR AM to read max-retries from the RM. (Zhijie Shen via
  1206. vinodkv)
  1207. MAPREDUCE-3829. [Gridmix] Gridmix should give better error message when
  1208. input data directory already exists and -generate opton is
  1209. given.(ravigummadi via tgraves)
  1210. MAPREDUCE-2722. [Gridmix] Gridmix simulated job's map's hdfsBytesRead
  1211. counter is wrong when compressed input is used.(ravigummadi via tgraves)
  1212. MAPREDUCE-3953. [Gridmix] Gridmix throws NPE and does not simulate a
  1213. job if the trace contains null taskStatus for a task. (ravigummadi via
  1214. tgraves)
  1215. MAPREDUCE-4087. [Gridmix] GenerateDistCacheData job of Gridmix can
  1216. become slow in some cases (ravigummadi via tgraves).
  1217. MAPREDUCE-5077. Remove mapreduce.util.ResourceCalculatorPlugin and related
  1218. code. (Karthik Kambatla via sseth)
  1219. MAPREDUCE-4083. [Gridmix] NPE in cpu emulation. (amarrk via tgraves)
  1220. MAPREDUCE-4100. [Gridmix] Bug fixed in compression emulation feature for
  1221. map only jobs. (amarrk via tgraves)
  1222. MAPREDUCE-4356. [Rumen] Provide access to the method
  1223. ParsedTask.obtainTaskAttempts(). (ravigummadi via tgraves)
  1224. MAPREDUCE-4149. [Rumen] Rumen fails to parse certain counter
  1225. strings. (ravigummadi via tgraves)
  1226. MAPREDUCE-3757. [Rumen] Fixed Rumen Folder to adjust shuffleFinished and
  1227. sortFinished times when needed. (Ravi Gummadi via tgraves)
  1228. MAPREDUCE-5138. Fix LocalDistributedCacheManager after YARN-112. (Omkar Vinit
  1229. Joshi via vinodkv)
  1230. MAPREDUCE-5086. MR app master deletes staging dir when sent a reboot
  1231. command from the RM. (Jian He via jlowe)
  1232. MAPREDUCE-5113. Streaming input/output types are ignored with java
  1233. mapper/reducer. (sandyr via tucu)
  1234. MAPREDUCE-5098. Fix findbugs warnings in gridmix. (kkambatl via tucu)
  1235. MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error
  1236. (Thomas Graves via jlowe)
  1237. MAPREDUCE-5136. TestJobImpl->testJobNoTasks fails with IBM JAVA (Amir
  1238. Sanjar via jlowe)
  1239. MAPREDUCE-5139. Update MR AM to use the modified startContainer API after
  1240. YARN-486. (Xuan Gong via vinodkv)
  1241. MAPREDUCE-5151. Update MR AM to use standard exit codes from the API after
  1242. YARN-444. (Sandy Ryza via vinodkv)
  1243. MAPREDUCE-5140. MR part of YARN-514 (Zhijie Shen via bikas)
  1244. MAPREDUCE-5128. mapred-default.xml is missing a bunch of history server
  1245. configs. (sandyr via tucu)
  1246. MAPREDUCE-4898. FileOutputFormat.checkOutputSpecs and
  1247. FileOutputFormat.setOutputPath incompatible with MR1. (rkanter via tucu)
  1248. MAPREDUCE-5078. TestMRAppMaster fails on Windows due to mismatched path
  1249. separators. (Chris Nauroth via sseth)
  1250. MAPREDUCE-4932. mapreduce.job#getTaskCompletionEvents incompatible with
  1251. Hadoop 1. (rkanter via tucu)
  1252. MAPREDUCE-5163. Update MR App to not use API utility methods for collections
  1253. after YARN-441. (Xuan Gong via vinodkv)
  1254. MAPREDUCE-5066. Added a timeout for the job.end.notification.url. (Ivan
  1255. Mitic via acmurthy)
  1256. MAPREDUCE-5146. application classloader may be used too early to load
  1257. classes. (Sangjin Lee via tomwhite)
  1258. MAPREDUCE-4737. Ensure that mapreduce APIs are semantically consistent
  1259. with mapred API w.r.t Mapper.cleanup and Reducer.cleanup; in the sense that
  1260. cleanup is now called even if there is an error. The old mapred API
  1261. already ensures that Mapper.close and Reducer.close are invoked during
  1262. error handling. Note that it is an incompatible change, however end-users
  1263. can override Mapper.run and Reducer.run to get the old (inconsistent)
  1264. behaviour. (acmurthy)
  1265. MAPREDUCE-5166. Fix ConcurrentModificationException due to insufficient
  1266. synchronization on updates to task Counters. (Sandy Ryza via acmurthy)
  1267. MAPREDUCE-5181. RMCommunicator should not use AMToken from the env.
  1268. (Vinod Kumar Vavilapalli via sseth)
  1269. MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after
  1270. YARN-577. (Hitesh Shah via vinodkv)
  1271. MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API
  1272. for the container. (Jian He via vinodkv)
  1273. MAPREDUCE-5179. Fix unit test in TestHSWebServices which fails when
  1274. versionInfo has parantheses like when running on a git checkout. (Hitesh Shah
  1275. via vinodkv)
  1276. MAPREDUCE-5193. A few MR tests use block sizes which are smaller than the
  1277. default minimum block size. (Andrew Wang via atm)
  1278. MAPREDUCE-5205. Fixed MR App to load tokens correctly. (vinodkv)
  1279. MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in
  1280. MR app after YARN-629. (Xuan Gong via vinodkv)
  1281. MAPREDUCE-5209. Fix units in a ShuffleScheduler log message.
  1282. (Tsuyoshi OZAWA via cdouglas)
  1283. MAPREDUCE-5212. Handling YarnRemoteException separately from IOException in
  1284. MR App's use of ClientRMProtocol after YARN-631. (Xuan Gong via vinodkv)
  1285. MAPREDUCE-5226. Handling YarnRemoteException separately from IOException in
  1286. MR App's use of AMRMProtocol after YARN-630. (Xuan Gong via vinodkv)
  1287. MAPREDUCE-4942. mapreduce.Job has a bunch of methods that throw
  1288. InterruptedException so its incompatible with MR1. (rkanter via tucu)
  1289. MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after
  1290. YARN-634. (Siddharth Seth via vinodkv)
  1291. MAPREDUCE-5208. Modified ShuffleHandler to use SecureIOUtils for reading
  1292. local files. (Omkar Vinit Joshi via vinodkv)
  1293. MAPREDUCE-5220. Setter methods in TaskCompletionEvent are public in MR1 and
  1294. protected in MR2. (sandyr via tucu)
  1295. MAPREDUCE-5240. Fix a bug in MRAppMaster because of which OutputCommitter
  1296. could not access credentials set by the user. (vinodkv)
  1297. MAPREDUCE-5244. Two functions changed their visibility in JobStatus.
  1298. (zjshen via tucu)
  1299. MAPREDUCE-4927. Historyserver 500 error due to NPE when accessing specific
  1300. counters page for failed job. (Ashwin Shankar via jlowe)
  1301. MAPREDUCE-5257. Fix issues in TestContainerLauncherImpl after YARN-617.
  1302. (Omkar Vinit Joshi via vinodkv)
  1303. MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after
  1304. YARN-716. (Siddharth Seth via vinodkv)
  1305. MAPREDUCE-5286. Change MapReduce to use ContainerTokenIdentifier instead
  1306. of the entire Container in the startContainer call - YARN-684.
  1307. (Vinod Kumar Vavilapalli via sseth)
  1308. MAPREDUCE-5299. Fix backward incompatibility for TaskCompletionEvent by
  1309. adding back setTaskID. (Zhijie Shen via acmurthy)
  1310. MAPREDUCE-5296. Fix backward incompatibility for JobControl by adding the
  1311. omitted addJob. (Zhijie Shen via acmurthy)
  1312. MAPREDUCE-5245. Added back constants to JobConf to fix incompatibilities.
  1313. (Zhijie Shen via acmurthy)
  1314. MAPREDUCE-5297. Updated MR App since BuilderUtils is no longer public
  1315. after YARN-748. (Jian He via vinodkv)
  1316. MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming
  1317. YarnRemoteException to YarnException. (Siddharth Seth via vinodkv)
  1318. MAPREDUCE-5308. Shuffling to memory can get out-of-sync when fetching
  1319. multiple compressed map outputs (Nathan Roberts via jlowe)
  1320. MAPREDUCE-5315. DistCp reports success even on failure. (mithun and jlowe
  1321. via daryn)
  1322. MAPREDUCE-5259. TestTaskLog fails on Windows because of path separators
  1323. missmatch. (Ivan Mitic via cnauroth)
  1324. MAPREDUCE-4019. -list-attempt-ids is not working (Ashwin Shankar,
  1325. Devaraj K, and B Anil Kumar via jlowe)
  1326. MAPREDUCE-5334. Fix failing unit tests - TestContainerLauncher,
  1327. TestContainerLauncherImpl. (Vinod Kumar Vavilapalli via sseth)
  1328. MAPREDUCE-5325. MR changes related to YARN-727. ClientRMProtocol.getAllApplications
  1329. should accept ApplicationType as a parameter. (Xuan Gong via hitesh)
  1330. MAPREDUCE-5291. Change MR App to use updated property names in
  1331. container-log4j.properties. (Zhijie Shen via sseth)
  1332. MAPREDUCE-5303. Changed MR app after moving ProtoBase to package impl.pb via
  1333. YARN-724. (Jian He via vinodkv)
  1334. MAPREDUCE-5312. TestRMNMInfo is failing. (sandyr via tucu)
  1335. MAPREDUCE-5304. mapreduce.Job killTask/failTask/getTaskCompletionEvents
  1336. methods have incompatible signature changes. (kkambatl via tucu)
  1337. MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle.
  1338. (Steve Loughran via vinodkv)
  1339. MAPREDUCE-5319. Set user.name in job.xml. (Xuan Gong via acmurthy)
  1340. MAPREDUCE-5310. MRAM should not normalize allocation request capabilities.
  1341. (tucu)
  1342. MAPREDUCE-5213. Re-assess TokenCache methods marked @Private.
  1343. (kkambatl via tucu)
  1344. MAPREDUCE-5412. Update MR app to use multiple containers API of
  1345. ContainerManager after YARN-926. (Jian He via vinodkv)
  1346. MAPREDUCE-5421. Fixed TestNonExistentJob failure after YARN-873. (Junping Du
  1347. via vinodkv)
  1348. MAPREDUCE-5419. TestSlive is getting FileNotFound Exception (Robert Parker
  1349. via jlowe)
  1350. MAPREDUCE-5399. Unnecessary Configuration instantiation in IFileInputStream
  1351. slows down merge. (Stanislav Barton via Sandy Ryza)
  1352. BREAKDOWN OF HADOOP-8562 SUBTASKS
  1353. MAPREDUCE-4739. Some MapReduce tests fail to find winutils.
  1354. (Chris Nauroth via suresh)
  1355. MAPREDUCE-4780. MapReduce distribution build fails on Windows.
  1356. (Chris Nauroth via suresh)
  1357. MAPREDUCE-4790. MapReduce build script would be more readable using abspath.
  1358. (Chris Nauroth via suresh)
  1359. MAPREDUCE-4869. Fix TestMapReduceChildJVM. (Chris Nauroth via acmurthy)
  1360. MAPREDUCE-4870. Fix TestMRJobsWithHistoryService. (Chris Nauroth via acmurthy)
  1361. MAPREDUCE-4983. Fixed various platform specific assumptions in various tests,
  1362. so that they can pass on Windows too. (Chris Nauroth via vinodkv)
  1363. HADOOP-9372. Fix bad timeout annotations on tests.
  1364. (Arpit Agarwal via suresh)
  1365. MAPREDUCE-4885. Streaming tests have multiple failures on Windows. (Chris
  1366. Nauroth via bikas)
  1367. MAPREDUCE-5177. Use common utils FileUtil#setReadable/Writable/Executable &
  1368. FileUtil#canRead/Write/Execute. (Ivan Mitic via suresh)
  1369. MAPREDUCE-5349. TestClusterMapReduceTestCase and TestJobName fail on Windows
  1370. in branch-2. (Chuan Liu via cnauroth)
  1371. MAPREDUCE-5355. MiniMRYarnCluster with localFs does not work on Windows.
  1372. (Chuan Liu via cnauroth)
  1373. MAPREDUCE-5359. JobHistory should not use File.separator to match timestamp
  1374. in path. (Chuan Liu via cnauroth)
  1375. MAPREDUCE-5357. Job staging directory owner checking could fail on Windows.
  1376. (Chuan Liu via cnauroth)
  1377. MAPREDUCE-5360. TestMRJobClient fails on Windows due to path format.
  1378. (Chuan Liu via cnauroth)
  1379. MAPREDUCE-5366. TestMRAsyncDiskService fails on Windows. (Chuan Liu via
  1380. cnauroth)
  1381. MAPREDUCE-5187. Create mapreduce command scripts on Windows. (Chuan Liu via
  1382. cnauroth)
  1383. MAPREDUCE-4374. Fix child task environment variable config and add support
  1384. for Windows. (Chuan Liu via cnauroth)
  1385. Release 2.0.6-alpha - 08/22/2013
  1386. INCOMPATIBLE CHANGES
  1387. NEW FEATURES
  1388. IMPROVEMENTS
  1389. OPTIMIZATIONS
  1390. Release 2.0.5-alpha - 06/06/2013
  1391. INCOMPATIBLE CHANGES
  1392. NEW FEATURES
  1393. IMPROVEMENTS
  1394. OPTIMIZATIONS
  1395. BUG FIXES
  1396. MAPREDUCE-5240 inside of FileOutputCommitter the initialized Credentials cache
  1397. appears to be empty. (vinodkv)
  1398. Release 2.0.4-alpha - 2013-04-25
  1399. INCOMPATIBLE CHANGES
  1400. NEW FEATURES
  1401. IMPROVEMENTS
  1402. OPTIMIZATIONS
  1403. BUG FIXES
  1404. MAPREDUCE-5006. Fix failing streaming tests due to MAPREDUCE-4994.
  1405. (Sandy Ryza via tomwhite)
  1406. MAPREDUCE-5088. MR Client gets an renewer token exception while Oozie is
  1407. submitting a job (Daryn Sharp via cos)
  1408. MAPREDUCE-5117. Changed MRClientProtocolPBClientImpl to be closeable and thus
  1409. fix failures in renewal of HistoryServer's delegations tokens. (Siddharth
  1410. Seth via vinodkv)
  1411. MAPREDUCE-5083. MiniMRCluster should use a random component when creating an
  1412. actual cluster (Siddharth Seth via hitesh)
  1413. MAPREDUCE-5094. Disabled memory monitoring by default in MiniMRYarnCluster
  1414. to avoid some downstream tests failing. (Siddharth Seth via vinodkv)
  1415. Release 2.0.3-alpha - 2013-02-06
  1416. INCOMPATIBLE CHANGES
  1417. MAPREDUCE-4123. Remove the 'mapred groups' command, which is no longer
  1418. supported. (Devaraj K via sseth)
  1419. MAPREDUCE-4938. Use token request messages defined in hadoop common.
  1420. (suresh)
  1421. NEW FEATURES
  1422. MAPREDUCE-4520. Added support for MapReduce applications to request for
  1423. CPU cores along-with memory post YARN-2. (acmurthy)
  1424. MAPREDUCE-4810. Added new admin command options for MR AM. (Jerry Chen via
  1425. vinodkv)
  1426. MAPREDUCE-4049. Experimental api to allow for alternate shuffle plugins.
  1427. (Avner BenHanoch via acmurthy)
  1428. MAPREDUCE-4807. Allow MapOutputBuffer to be pluggable. (masokan via tucu)
  1429. MAPREDUCE-4808. Refactor MapOutput and MergeManager to facilitate reuse
  1430. by Shuffle implementations. (masokan via tucu)
  1431. IMPROVEMENTS
  1432. MAPREDUCE-3678. The Map tasks logs should have the value of input
  1433. split it processed. (harsh)
  1434. MAPREDUCE-4616. Improve javadoc for MultipleOutputs. (Tony Burton via
  1435. acmurthy)
  1436. HADOOP-8911. CRLF characters in source and text files.
  1437. (Raja Aluri via suresh)
  1438. MAPREDUCE-4723. Fix warnings found by findbugs 2. (Sandy Ryza via eli)
  1439. MAPREDUCE-4703. Add the ability to start the MiniMRClientCluster using
  1440. the configurations used before it is being stopped. (ahmed.radwan via tucu)
  1441. MAPREDUCE-4845. ClusterStatus.getMaxMemory() and getUsedMemory() exist in
  1442. MR1 but not MR2. (Sandy Ryza via tomwhite)
  1443. MAPREDUCE-4899. Implemented a MR specific plugin for tracking finished
  1444. applications that YARN's ResourceManager doesn't keep track off anymore
  1445. (Derek Dagit via vinodkv)
  1446. MAPREDUCE-4920. Use security token protobuf definition from hadoop common.
  1447. (Suresh Srinivas via vinodkv)
  1448. MAPREDUCE-4907. TrackerDistributedCacheManager issues too many getFileStatus
  1449. calls. (sandyr via tucu)
  1450. MAPREDUCE-4949. Enable multiple pi jobs to run in parallel. (sandyr via tucu)
  1451. MAPREDUCE-4809. Change visibility of classes for pluggable sort changes.
  1452. (masokan via tucu)
  1453. MAPREDUCE-4838. Add additional fields like Locality, Avataar to the
  1454. JobHistory logs. (Zhijie Shen via sseth)
  1455. MAPREDUCE-4971. Minor extensibility enhancements to Counters &
  1456. FileOutputFormat. (Arun C Murthy via sseth)
  1457. MAPREDUCE-4977. Documentation for pluggable shuffle and pluggable sort.
  1458. (tucu)
  1459. OPTIMIZATIONS
  1460. MAPREDUCE-4893. Fixed MR ApplicationMaster to do optimal assignment of
  1461. containers to get maximum locality. (Bikas Saha via vinodkv)
  1462. BUG FIXES
  1463. MAPREDUCE-4272. SortedRanges.Range#compareTo is not spec compliant.
  1464. (Yu Gao via llu)
  1465. MAPREDUCE-4607. Race condition in ReduceTask completion can result in Task
  1466. being incorrectly failed. (Bikas Saha via tomwhite)
  1467. MAPREDUCE-4646. Fixed MR framework to send diagnostic information correctly
  1468. to clients in case of failed jobs also. (Jason Lowe via vinodkv)
  1469. MAPREDUCE-4674. Hadoop examples secondarysort has a typo
  1470. "secondarysrot" in the usage. (Robert Justice via eli)
  1471. MAPREDUCE-4681. Fix unit tests broken by HDFS-3910. (acmurthy)
  1472. MAPREDUCE-4712. mr-jobhistory-daemon.sh doesn't accept --config
  1473. (Vinod Kumar Vavilapalli via tgraves)
  1474. MAPREDUCE-4654. TestDistCp is ignored. (Sandy Ryza via tomwhite)
  1475. MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and
  1476. UNASSIGNED states. (Mayank Bansal via sseth)
  1477. MAPREDUCE-1806. CombineFileInputFormat does not work with paths not on default FS. (Gera Shegalov via tucu)
  1478. MAPREDUCE-4777. In TestIFile, testIFileReaderWithCodec relies on
  1479. testIFileWriterWithCodec. (Sandy Ryza via tomwhite)
  1480. MAPREDUCE-4800. Cleanup o.a.h.mapred.MapTaskStatus - remove unused
  1481. code. (kkambatl via tucu)
  1482. MAPREDUCE-4861. Cleanup: Remove unused mapreduce.security.token.DelegationTokenRenewal.
  1483. (kkambatl via tucu)
  1484. MAPREDUCE-4856. TestJobOutputCommitter uses same directory as
  1485. TestJobCleanup. (Sandy Ryza via tomwhite)
  1486. MAPREDUCE-4895. Fix compilation failure of org.apache.hadoop.mapred.
  1487. gridmix.TestResourceUsageEmulators (Dennis Y via tgraves)
  1488. MAPREDUCE-4278. Cannot run two local jobs in parallel from the same
  1489. gateway. (Sandy Ryza via tomwhite)
  1490. MAPREDUCE-1700. User supplied dependencies may conflict with MapReduce
  1491. system JARs. (tomwhite)
  1492. MAPREDUCE-4936. JobImpl uber checks for cpu are wrong (Arun C Murthy via
  1493. jlowe)
  1494. MAPREDUCE-4924. flakey test: org.apache.hadoop.mapred.TestClusterMRNotification.testMR.
  1495. (rkanter via tucu)
  1496. MAPREDUCE-4923. Add toString method to TaggedInputSplit. (sandyr via tucu)
  1497. MAPREDUCE-4948. Fix a failing unit test TestYARNRunner.testHistoryServerToken.
  1498. (Junping Du via sseth)
  1499. MAPREDUCE-4803. Remove duplicate copy of TestIndexCache. (Mariappan Asokan
  1500. via sseth)
  1501. MAPREDUCE-2264. Job status exceeds 100% in some cases.
  1502. (devaraj.k and sandyr via tucu)
  1503. MAPREDUCE-4969. TestKeyValueTextInputFormat test fails with Open JDK 7.
  1504. (Arpit Agarwal via suresh)
  1505. MAPREDUCE-4884. Streaming tests fail to start MiniMRCluster due to missing
  1506. queue configuration. (Chris Nauroth via suresh)
  1507. MAPREDUCE-4953. HadoopPipes misuses fprintf. (Andy Isaacson via atm)
  1508. Release 2.0.2-alpha - 2012-09-07
  1509. INCOMPATIBLE CHANGES
  1510. NEW FEATURES
  1511. MAPREDUCE-987. Exposing MiniDFS and MiniMR clusters as a single process
  1512. command-line. (ahmed via tucu)
  1513. MAPREDUCE-4417. add support for encrypted shuffle (tucu)
  1514. MAPREDUCE-4355. Add RunningJob.getJobStatus() (kkambatl via tucu)
  1515. MAPREDUCE-3451. Port Fair Scheduler to MR2 (pwendell via tucu)
  1516. MAPREDUCE-4438. Add a simple, generic client to run 'easy' AMs in YARN.
  1517. (Bikas Saha via acmurthy)
  1518. IMPROVEMENTS
  1519. MAPREDUCE-4157. ResourceManager should not kill apps that are well behaved
  1520. (Jason Lowe via bobby)
  1521. MAPREDUCE-4511. Add IFile readahead (ahmed via tucu)
  1522. MAPREDUCE-4408. allow jobs to set a JAR that is in the distributed cached
  1523. (rkanter via tucu)
  1524. MAPREDUCE-4440. Changed SchedulerApp and SchedulerNode to be a minimal
  1525. interface to allow schedulers to maintain their own. (acmurthy)
  1526. MAPREDUCE-4146. Support limits on task status string length and number of
  1527. block locations in branch-2. (Ahmed Radwan via tomwhite)
  1528. MAPREDUCE-3871. Allow symlinking in LocalJobRunner DistributedCache.
  1529. (tomwhite)
  1530. MAPREDUCE-3921. MR AM should act on node health status changes.
  1531. (Bikas Saha via sseth)
  1532. MAPREDUCE-4355. Add RunningJob.getJobStatus() (kkambatl via tucu)
  1533. MAPREDUCE-4427. Added an 'unmanaged' mode for AMs so as to ease
  1534. development of new applications. (Bikas Saha via acmurthy)
  1535. MAPREDUCE-3289. Make use of fadvise in the NM's shuffle handler.
  1536. (Todd Lipcon and Siddharth Seth via sseth)
  1537. MAPREDUCE-4580. Change MapReduce to use the yarn-client module.
  1538. (Vinod Kumar Vavilapalli via sseth)
  1539. MAPREDUCE-4579. Split TestTaskAttempt into two so as to pass tests on
  1540. jdk7. (Thomas Graves via vinodkv)
  1541. MAPREDUCE-4638. MR AM supplies MapReduce jars in classpath rather than
  1542. rely on YARN. (acmurthy)
  1543. BUG FIXES
  1544. MAPREDUCE-4422. YARN_APPLICATION_CLASSPATH needs a documented default value in
  1545. YarnConfiguration. (ahmed via tucu)
  1546. MAPREDUCE-4406. Users should be able to specify the MiniCluster ResourceManager
  1547. and JobHistoryServer ports. (ahmed via tucu)
  1548. MAPREDUCE-4407. Add hadoop-yarn-server-tests-<version>-tests.jar to hadoop dist
  1549. package. (ahmed via tucu)
  1550. MAPREDUCE-4465. Update description of yarn.nodemanager.address property.
  1551. (bowang via tucu)
  1552. MAPREDUCE-4342. Distributed Cache gives inconsistent result if cache files
  1553. get deleted from tasktracker. (mayank_bansal via tucu)
  1554. MAPREDUCE-4498. Remove hsqldb jar from Hadoop runtime classpath. (rkanter via tucu)
  1555. MAPREDUCE-4494. TestFifoScheduler failing with Metrics source QueueMetrics,q0=default
  1556. already exists!. (ahmed.radwan via tucu)
  1557. MAPREDUCE-4484. Incorrect IS_MINI_YARN_CLUSTER property name in YarnConfiguration.
  1558. (ahmed.radwan via tucu)
  1559. MAPREDUCE-4562. Support for "FileSystemCounter" legacy counter group name
  1560. for compatibility reasons is creating incorrect counter name.
  1561. (Jarek Jarcec Cecho via tomwhite)
  1562. MAPREDUCE-4068. Jars in lib subdirectory of the submittable JAR are not added to the
  1563. classpath (rkanter via tucu)
  1564. MAPREDUCE-4577. HDFS-3672 broke
  1565. TestCombineFileInputFormat.testMissingBlocks() test (atm)
  1566. MAPREDUCE-4470. Fix TestCombineFileInputFormat.testForEmptyFile (ikatsov via tucu)
  1567. MAPREDUCE-4608. hadoop-mapreduce-client is missing some dependencies.
  1568. (tucu via tomwhite)
  1569. MAPREDUCE-4610. Support deprecated mapreduce.job.counters.limit property in
  1570. MR2. (tomwhite)
  1571. MAPREDUCE-4629. Remove JobHistory.DEBUG_MODE (Karthik Kambatla via bobby)
  1572. MAPREDUCE-4642. MiniMRClientClusterFactory should not use job.setJar() (rkanter via tucu)
  1573. MAPREDUCE-4148. MapReduce should not have a compile-time dependency on
  1574. HDFS. (tomwhite)
  1575. MAPREDUCE-4250. hadoop-config.sh missing variable exports, causes Yarn
  1576. jobs to fail with ClassNotFoundException MRAppMaster. (phunt via tucu)
  1577. MAPREDUCE-4002. MultiFileWordCount job fails if the input path is not
  1578. from default file system. (Bhallamudi Venkata Siva Kamesh via todd)
  1579. MAPREDUCE-4274 MapOutputBuffer should use native byte order for kvmeta.
  1580. (todd via bobby)
  1581. MAPREDUCE-4262. NM gives wrong log message saying "Connected to
  1582. ResourceManager" before trying to connect. (Devaraj K via tgraves)
  1583. MAPREDUCE-4276. Allow setting yarn.nodemanager.delete.debug-delay-sec
  1584. property to "-1" for easier container debugging. (ahmed via tucu)
  1585. MAPREDUCE-4224. TestFifoScheduler throws
  1586. org.apache.hadoop.metrics2.MetricsException (Devaraj K via tgraves)
  1587. MAPREDUCE-3493. Add the default mapreduce.shuffle.port property
  1588. to mapred-default.xml (Madhukara Phatak via harsh)
  1589. MAPREDUCE-4307. TeraInputFormat calls FileSystem.getDefaultBlockSize()
  1590. without a Path - Failure when using ViewFileSystem. (Ahmed Radwan via eli)
  1591. MAPREDUCE-4313. TestTokenCache doesn't compile due
  1592. TokenCache.getDelegationToken compilation error (bobby)
  1593. MAPREDUCE-3873. Fixed NodeManagers' decommissioning at RM to accept IP
  1594. addresses also. (xieguiming via vinodkv)
  1595. MAPREDUCE-4306. Fix distributed shell to work with users other than the one
  1596. running the daemons. (Ahmed Radwan via sseth)
  1597. MAPREDUCE-4031. Prevent a Node Manager hang during shutdown.
  1598. (Devaraj K via sseth)
  1599. MAPREDUCE-4336. Distributed Shell fails when used with the CapacityScheduler
  1600. (ahmed via tucu)
  1601. MAPREDUCE-4290. Fix Yarn Applicaiton Status to MR JobState conversion.
  1602. (Devaraj K via sseth)
  1603. MAPREDUCE-2289. Permissions race can make getStagingDir fail on local filesystem
  1604. (ahmed via tucu)
  1605. MAPREDUCE-4372. Deadlock in Resource Manager (Devaraj K via bobby)
  1606. MAPREDUCE-4376. TestClusterMRNotification times out (Kihwal Lee via bobby)
  1607. MAPREDUCE-4383. HadoopPipes.cc needs to include unistd.h.
  1608. (Andy Isaacson via eli)
  1609. MAPREDUCE-2739. Update installation docs (remove YarnClientFactory) (bowang via tucu)
  1610. MAPREDUCE-3993. Graceful handling of codec errors during decompression
  1611. (kkambatl via tucu)
  1612. MAPREDUCE-4416. Some tests fail if Clover is enabled (Kihwal Lee via bobby)
  1613. MAPREDUCE-4441. Fix build issue caused by MR-3451 (kkambatl via tucu)
  1614. HADOOP-8499. Lower min.user.id to 500 for the tests.
  1615. (Colin Patrick McCabe via eli)
  1616. MAPREDUCE-4395. Possible NPE at ClientDistributedCacheManager
  1617. #determineTimestamps (Bhallamudi via bobby)
  1618. MAPREDUCE-4380. Empty Userlogs directory is getting created under logs
  1619. directory (Devaraj K via bobby)
  1620. MAPREDUCE-4649. Ensure MapReduce JobHistory Daemon doens't assume
  1621. HADOOP_YARN_HOME and HADOOP_MAPRED_HOME are the same. (vinodkv via
  1622. acmurthy)
  1623. Release 2.0.0-alpha - 05-23-2012
  1624. INCOMPATIBLE CHANGES
  1625. MAPREDUCE-3545. Remove Avro RPC. (suresh)
  1626. NEW FEATURES
  1627. IMPROVEMENTS
  1628. MAPREDUCE-2887. Due to HADOOP-7524, change RPC to allow multiple protocols
  1629. including multuple versions of the same protocol (Sanjay Radia)
  1630. MAPREDUCE-2934. MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup
  1631. process (atm)
  1632. HADOOP-7862. MR changes to work with HADOOP 7862: Move the support for
  1633. multiple protocols to lower layer so that Writable, PB and Avro can all
  1634. use it (Sanjay Radia)
  1635. MAPREDUCE-3909 Javadoc the Service interfaces (stevel)
  1636. MAPREDUCE-3885. Avoid an unnecessary copy for all requests/responses in
  1637. MRs ProtoOverHadoopRpcEngine. (Devaraj Das via sseth)
  1638. MAPREDUCE-3935. Annotate Counters.Counter and Counters.Group as @Public.
  1639. (tomwhite)
  1640. MAPREDUCE-3991. Streaming FAQ has some wrong instructions about input files
  1641. splitting. (harsh)
  1642. MAPREDUCE-3773. Add queue metrics with buckets for job run times. (omalley
  1643. via acmurthy)
  1644. MAPREDUCE-3970. Add ServiceOperations class to aid working with Services
  1645. (stevel)
  1646. MAPREDUCE-3353. Add a channel between RM and AM to get information on
  1647. nodes. (Bikas Saha via acmurthy)
  1648. MAPREDUCE-3955. Change MR to use ProtobufRpcEngine from hadoop-common
  1649. instead of ProtoOverHadoopRpcEngine. (Jitendra Nath Pandey via sseth)
  1650. MAPREDUCE-4103. Fix HA docs for changes to shell command fencer args (todd)
  1651. MAPREDUCE-4093. Improve RM WebApp start up when proxy address is not set
  1652. (Devaraj K vai bobby)
  1653. MAPREDUCE-4138. Reduce memory usage of counters due to non-static nested
  1654. classes. (tomwhite)
  1655. MAPREDUCE-3883. Document yarn.nodemanager.delete.debug-delay-sec
  1656. configuration property (Eugene Koontz via tgraves)
  1657. MAPREDUCE-4219. make default container-executor.conf.dir be a path
  1658. relative to the container-executor binary. (rvs via tucu)
  1659. MAPREDUCE-4205. retrofit all JVM shutdown hooks to use ShutdownHookManager
  1660. (tucu)
  1661. HADOOP-8285 MR changes for Use ProtoBuf for RpcPayLoadHeader (sanjay radia)
  1662. MAPREDUCE-2220. Fix new API FileOutputFormat-related typos in
  1663. mapred-default.xml (Rui Kubo via harsh)
  1664. MAPREDUCE-3907. Document entries mapred-default.xml for the
  1665. jobhistory server. (Eugene Koontz via harsh)
  1666. MAPREDUCE-3906. Fix inconsistency in documentation regarding
  1667. mapreduce.jobhistory.principal. (Eugene Koontz via harsh)
  1668. MAPREDUCE-4432. Confusing warning message when GenericOptionsParser
  1669. is not used. (Gabriel Reid via harsh)
  1670. OPTIMIZATIONS
  1671. BUG FIXES
  1672. MAPREDUCE-3740. Fixed broken mapreduce compilation after the patch for
  1673. HADOOP-7965. (Devaraj K via vinodkv)
  1674. MAPREDUCE-3818. Fixed broken compilation in TestSubmitJob after the patch
  1675. for HDFS-2895. (Suresh Srinivas via vinodkv)
  1676. MAPREDUCE-2942. TestNMAuditLogger.testNMAuditLoggerWithIP failing (Thomas
  1677. Graves via mahadev)
  1678. MAPREDUCE-3933. Failures because MALLOC_ARENA_MAX is not set (ahmed via tucu)
  1679. MAPREDUCE-3728. ShuffleHandler can't access results when configured in a
  1680. secure mode (ahmed via tucu)
  1681. MAPREDUCE-3952. In MR2, when Total input paths to process == 1,
  1682. CombinefileInputFormat.getSplits() returns 0 split. (zhenxiao via tucu)
  1683. MAPREDUCE-3578. Starting nodemanager as root gives "Unknown -jvm option"
  1684. (tomwhite)
  1685. MAPREDUCE-3348. Fixed a bug in MR client to redirect to JobHistoryServer
  1686. correctly when RM forgets the app. (Devaraj K via vinodkv)
  1687. MAPREDUCE-3974. TestSubmitJob in MR1 tests doesn't compile after HDFS-162
  1688. merge. (atm)
  1689. MAPREDUCE-4007. JobClient getJob(JobID) should return NULL if the job
  1690. does not exist (for backwards compatibility) (tucu)
  1691. MAPREDUCE-3431 NPE in Resource Manager shutdown. (stevel)
  1692. MAPREDUCE-4010. TestWritableJobConf fails on trunk (tucu via bobby)
  1693. MAPREDUCE-3992. Reduce fetcher doesn't verify HTTP status code of response
  1694. (todd)
  1695. MAPREDUCE-4066. Use default value when fetching MR_AM_STAGING_DIR
  1696. (xieguiming via harsh)
  1697. MAPREDUCE-3377. Added a unit test to ensure OutputCommitter.checkOutputSpecs
  1698. is called prior to copying job.xml. (Jane Chen via acmurthy)
  1699. MAPREDUCE-4081. TestMROutputFormat.java does not compile (Jason Lowe via
  1700. bobby)
  1701. MAPREDUCE-4082. hadoop-mapreduce-client-app's mrapp-generated-classpath
  1702. file should not be in the module JAR (tucu)
  1703. MAPREDUCE-3916. various issues with running yarn proxyserver (devaraj via tucu)
  1704. MAPREDUCE-4091. tools testcases failing because of MAPREDUCE-4082 (tucu)
  1705. MAPREDUCE-4098. TestMRApps testSetClasspath fails (tucu)
  1706. MAPREDUCE-4097. tools testcases fail because missing mrapp-generated-classpath
  1707. file in classpath (rvs via tucu)
  1708. MAPREDUCE-4113. Fix tests org.apache.hadoop.mapred.TestClusterMRNotification
  1709. (Devaraj K via bobby)
  1710. MAPREDUCE-4112. Fix tests org.apache.hadoop.mapred.TestClusterMapReduceTestCase
  1711. (Devaraj K via bobby)
  1712. MAPREDUCE-4111. Fix tests in org.apache.hadoop.mapred.TestJobName (Devaraj
  1713. K via bobby)
  1714. MAPREDUCE-4110. Fix tests in org.apache.hadoop.mapred.TestMiniMRClasspath &
  1715. org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers (Devaraj K via
  1716. bobby)
  1717. MAPREDUCE-4105. Yarn RackResolver ignores rack configurations.
  1718. (Ahmed Radwan via tomwhite)
  1719. MAPREDUCE-3869. Fix classpath for DistributedShell application. (Devaraj K
  1720. via sseth)
  1721. MAPREDUCE-4057. Update RAID for the HA and fsdataset changes. (Devaraj K
  1722. via szetszwo)
  1723. MAPREDUCE-4076. Stream job fails with ZipException when use yarn jar
  1724. command (Devaraj K via bobby)
  1725. MAPREDUCE-4108. Fix tests in org.apache.hadoop.util.TestRunJar
  1726. (Devaraj K via tgraves)
  1727. MAPREDUCE-4107. Fix tests in org.apache.hadoop.ipc.TestSocketFactory
  1728. (Devaraj K via tgraves)
  1729. MAPREDUCE-4147. YARN should not have a compile-time dependency on HDFS.
  1730. (tomwhite)
  1731. MAPREDUCE-4008. ResourceManager throws MetricsException on start up
  1732. saying QueueMetrics MBean already exists (Devaraj K via tgraves)
  1733. MAPREDUCE-3867. MiniMRYarn/MiniYarn uses fixed ports (tucu)
  1734. MAPREDUCE-4141. clover integration broken, also mapreduce poms are
  1735. pulling in clover as a dependency. (phunt via tucu)
  1736. MAPREDUCE-4193. broken doc link for yarn-default.xml in site.xml.
  1737. (phunt via tomwhite)
  1738. MAPREDUCE-4202. TestYarnClientProtocolProvider is broken (Daryn Sharp via
  1739. bobby)
  1740. MAPREDUCE-3173. MRV2 UI doesn't work properly without internet (Devaraj K
  1741. via bobby)
  1742. MAPREDUCE-3958. RM: Remove RMNodeState and replace it with NodeState
  1743. (Bikas Saha via bobby)
  1744. MAPREDUCE-4231. Update RAID to use the new BlockCollection interface.
  1745. (szetszwo)
  1746. MAPREDUCE-4483. 2.0 build does not work (John George via bobby)
  1747. MAPREDUCE-4444. nodemanager fails to start when one of the local-dirs is
  1748. bad (Jason Lowe via bobby)
  1749. Release 0.23.10 - UNRELEASED
  1750. INCOMPATIBLE CHANGES
  1751. NEW FEATURES
  1752. IMPROVEMENTS
  1753. MAPREDUCE-5640. Rename TestLineRecordReader in jobclient module (Jason Lowe
  1754. via jeagles)
  1755. MAPREDUCE-3191. docs for map output compression incorrectly reference
  1756. SequenceFile (Chen He via jeagles)
  1757. OPTIMIZATIONS
  1758. MAPREDUCE-1981. Improve getSplits performance by using listLocatedStatus
  1759. (Hairong Kuang and Jason Lowe via jlowe)
  1760. BUG FIXES
  1761. MAPREDUCE-3193. FileInputFormat doesn't read files recursively in the
  1762. input path dir (Devaraj K via jlowe)
  1763. MAPREDUCE-5380. Invalid mapred command should return non-zero exit code
  1764. (Stephen Chu via jlowe)
  1765. MAPREDUCE-5317. Stale files left behind for failed jobs (Ravi Prakash via
  1766. jlowe)
  1767. MAPREDUCE-5251. Reducer should not implicate map attempt if it has
  1768. insufficient space to fetch map output (Ashwin Shankar via jlowe)
  1769. MAPREDUCE-5419. TestSlive is getting FileNotFound Exception (Robert Parker
  1770. via jlowe)
  1771. MAPREDUCE-5440. TestCopyCommitter Fails on JDK7 (Robert Parker via jlowe)
  1772. MAPREDUCE-5001. LocalJobRunner has race condition resulting in job
  1773. failures (Sandy Ryza via jlowe)
  1774. MAPREDUCE-5475. MRClientService does not verify ACLs properly (jlowe)
  1775. MAPREDUCE-5504. mapred queue -info inconsistent with types (Kousuke Saruta
  1776. via tgraves)
  1777. MAPREDUCE-5513. ConcurrentModificationException in JobControl (Robert
  1778. Parker via jlowe)
  1779. MAPREDUCE-5586. TestCopyMapper#testCopyFailOnBlockSizeDifference fails when
  1780. run from hadoop-tools/hadoop-distcp directory (jeagles)
  1781. MAPREDUCE-5587. TestTextOutputFormat fails on JDK7 (jeagles)
  1782. MAPREDUCE-5373. TestFetchFailure.testFetchFailureMultipleReduces could fail
  1783. intermittently (jeagles)
  1784. Release 0.23.9 - 2013-07-08
  1785. INCOMPATIBLE CHANGES
  1786. NEW FEATURES
  1787. IMPROVEMENTS
  1788. MAPREDUCE-5084. fix coverage org.apache.hadoop.mapreduce.v2.app.webapp and
  1789. org.apache.hadoop.mapreduce.v2.hs.webapp (Aleksey Gorshkov via jeagles)
  1790. OPTIMIZATIONS
  1791. MAPREDUCE-5268. Improve history server startup performance (Karthik
  1792. Kambatla via jlowe)
  1793. BUG FIXES
  1794. MAPREDUCE-5308. Shuffling to memory can get out-of-sync when fetching
  1795. multiple compressed map outputs (Nathan Roberts via jlowe)
  1796. MAPREDUCE-5315. DistCp reports success even on failure. (mithun and jlowe
  1797. via daryn)
  1798. MAPREDUCE-4019. -list-attempt-ids is not working (Ashwin Shankar,
  1799. Devaraj K, and B Anil Kumar via jlowe)
  1800. MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal
  1801. task-state (Ashwin Shankar via jlowe)
  1802. Release 0.23.8 - 2013-06-05
  1803. INCOMPATIBLE CHANGES
  1804. NEW FEATURES
  1805. IMPROVEMENTS
  1806. MAPREDUCE-5065. DistCp should skip checksum comparisons if block-sizes
  1807. are different on source/target (Mithun Radhakrishnan via kihwal)
  1808. OPTIMIZATIONS
  1809. BUG FIXES
  1810. MAPREDUCE-5015. Coverage fix for org.apache.hadoop.mapreduce.tools.CLI
  1811. (Aleksey Gorshkov via tgraves)
  1812. MAPREDUCE-5147. Maven build should create
  1813. hadoop-mapreduce-client-app-VERSION.jar directly (Robert Parker via tgraves)
  1814. MAPREDUCE-4927. Historyserver 500 error due to NPE when accessing specific
  1815. counters page for failed job. (Ashwin Shankar via jlowe)
  1816. Release 0.23.7 - 2013-04-18
  1817. INCOMPATIBLE CHANGES
  1818. NEW FEATURES
  1819. IMPROVEMENTS
  1820. MAPREDUCE-4905. test org.apache.hadoop.mapred.pipes
  1821. (Aleksey Gorshkov via bobby)
  1822. MAPREDUCE-4989. JSONify DataTables input data for Attempts page (Ravi
  1823. Prakash via jlowe)
  1824. MAPREDUCE-5027. Shuffle does not limit number of outstanding connections
  1825. (Robert Parker via jeagles)
  1826. MAPREDUCE-4972. Coverage fixing for org.apache.hadoop.mapreduce.jobhistory
  1827. (Aleksey Gorshkov via bobby)
  1828. OPTIMIZATIONS
  1829. MAPREDUCE-4946. Fix a performance problem for large jobs by reducing the
  1830. number of map completion event type conversions. (Jason Lowe via sseth)
  1831. MAPREDUCE-4822. Unnecessary conversions in History Events. (Chu Tong via
  1832. jlowe)
  1833. BUG FIXES
  1834. MAPREDUCE-4458. Warn if java.library.path is used for AM or Task
  1835. (Robert Parker via jeagles)
  1836. MAPREDUCE-4992. AM hangs in RecoveryService when recovering tasks with
  1837. speculative attempts (Robert Parker via jlowe)
  1838. MAPREDUCE-5009. Killing the Task Attempt slated for commit does not clear
  1839. the value from the Task commitAttempt member (Robert Parker via jeagles)
  1840. MAPREDUCE-4871. AM uses mapreduce.jobtracker.split.metainfo.maxsize but
  1841. mapred-default has mapreduce.job.split.metainfo.maxsize (Jason Lowe via
  1842. jeagles)
  1843. MAPREDUCE-4794. DefaultSpeculator generates error messages on normal
  1844. shutdown (Jason Lowe via jeagles)
  1845. MAPREDUCE-5043. Fetch failure processing can cause AM event queue to
  1846. backup and eventually OOM (Jason Lowe via bobby)
  1847. MAPREDUCE-5023. History Server Web Services missing Job Counters (Ravi
  1848. Prakash via tgraves)
  1849. MAPREDUCE-5060. Fetch failures that time out only count against the first
  1850. map task (Robert Joseph Evans via jlowe)
  1851. MAPREDUCE-5042. Reducer unable to fetch for a map task that was recovered
  1852. (Jason Lowe via bobby)
  1853. MAPREDUCE-5053. java.lang.InternalError from decompression codec cause
  1854. reducer to fail (Robert Parker via jeagles)
  1855. MAPREDUCE-4991. coverage for gridmix (Aleksey Gorshkov via tgraves)
  1856. MAPREDUCE-5007. fix coverage org.apache.hadoop.mapreduce.v2.hs (Aleksey
  1857. Gorshkov via tgraves)
  1858. MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error
  1859. (Thomas Graves via jlowe)
  1860. Release 0.23.6 - 2013-02-06
  1861. INCOMPATIBLE CHANGES
  1862. NEW FEATURES
  1863. IMPROVEMENTS
  1864. MAPREDUCE-4811. JobHistoryServer should show when it was started in WebUI
  1865. About page (Ravi Prakash via jlowe)
  1866. OPTIMIZATIONS
  1867. BUG FIXES
  1868. MAPREDUCE-4802. Takes a long time to load the task list on the AM for
  1869. large jobs (Ravi Prakash via bobby)
  1870. MAPREDUCE-4764. repair TestBinaryTokenFile (Ivan A. Veselovsky via bobby)
  1871. MAPREDUCE-4825. JobImpl.finished doesn't expect ERROR as a final job state
  1872. (jlowe via bobby)
  1873. MAPREDUCE-4817. Hardcoded task ping timeout kills tasks localizing large
  1874. amounts of data (tgraves)
  1875. MAPREDUCE-4836. Elapsed time for running tasks on AM web UI tasks page is 0
  1876. (Ravi Prakash via jeagles)
  1877. MAPREDUCE-4842. Shuffle race can hang reducer (Mariappan Asokan via jlowe)
  1878. MAPREDUCE-4833. Task can get stuck in FAIL_CONTAINER_CLEANUP (Robert
  1879. Parker via jlowe)
  1880. MAPREDUCE-4793. Problem with adding resources when using both -files and
  1881. -file to hadoop streaming (jlowe)
  1882. MAPREDUCE-4890. Invalid TaskImpl state transitions when task fails while
  1883. speculating (jlowe)
  1884. MAPREDUCE-4902. Fix typo "receievd" should be "received" in log output
  1885. (Albert Chu via jlowe)
  1886. MAPREDUCE-4813. AM timing out during job commit (jlowe via bobby)
  1887. MAPREDUCE-4279. getClusterStatus() fails with null pointer exception when
  1888. running jobs in local mode (Devaraj K via bobby)
  1889. MAPREDUCE-4832. MR AM can get in a split brain situation (jlowe)
  1890. MAPREDUCE-4894. Renewal / cancellation of JobHistory tokens (Siddharth
  1891. Seth via tgraves)
  1892. MAPREDUCE-4819. AM can rerun job after reporting final job status to the
  1893. client (bobby and Bikas Saha via bobby)
  1894. MAPREDUCE-4913. TestMRAppMaster#testMRAppMasterMissingStaging occasionally
  1895. exits (Jason Lowe via tgraves)
  1896. MAPREDUCE-4848. TaskAttemptContext cast error during AM recovery (Jerry
  1897. Chen via jlowe)
  1898. MAPREDUCE-4921. JobClient should acquire HS token with RM principal
  1899. (daryn via bobby)
  1900. MAPREDUCE-4934. Maven RAT plugin is not checking all source files (tgraves)
  1901. MAPREDUCE-4678. Running the Pentomino example with defaults throws
  1902. java.lang.NegativeArraySizeException (Chris McConnell via harsh)
  1903. MAPREDUCE-4925. The pentomino option parser may be buggy.
  1904. (Karthik Kambatla via harsh)
  1905. Release 0.23.5 - 2012-11-28
  1906. INCOMPATIBLE CHANGES
  1907. NEW FEATURES
  1908. IMPROVEMENTS
  1909. MAPREDUCE-4596. Split StateMachine state from states seen by MRClientProtocol
  1910. for Job, Task and TaskAttempt. (Siddarth Seth via vinodkv)
  1911. MAPREDUCE-4752. Reduce MR AM memory usage through String Interning (Robert
  1912. Evans via tgraves)
  1913. MAPREDUCE-4266. remove Ant remnants from MR (tgraves via bobby)
  1914. MAPREDUCE-4666. JVM metrics for history server (jlowe via jeagles)
  1915. OPTIMIZATIONS
  1916. MAPREDUCE-4720. Browser thinks History Server main page JS is taking too
  1917. long (Ravi Prakash via bobby)
  1918. BUG FIXES
  1919. MAPREDUCE-4554. Job Credentials are not transmitted if security is turned
  1920. off (Benoy Antony via bobby)
  1921. MAPREDUCE-4705. Fix a bug in job history lookup, which makes older jobs
  1922. inaccessible despite the presence of a valid history file. (Jason Lowe
  1923. via sseth)
  1924. MAPREDUCE-4521. mapreduce.user.classpath.first incompatibility with 0.20/1.x
  1925. (Ravi Prakash via bobby)
  1926. MAPREDUCE-4721. Task startup time in JHS is same as job startup time.
  1927. (Ravi Prakash via bobby)
  1928. MAPREDUCE-4479. Fix parameter order in assertEquals() in
  1929. TestCombineInputFileFormat.java (Mariappan Asokan via bobby)
  1930. MAPREDUCE-4733. Reducer can fail to make progress during shuffle if too many
  1931. reducers complete consecutively. (Jason Lowe via vinodkv)
  1932. MAPREDUCE-4740. only .jars can be added to the Distributed Cache
  1933. classpath. (Robert Joseph Evans via jlowe)
  1934. MAPREDUCE-4229. Intern counter names in the JT (Miomir Boljanovic and bobby via daryn)
  1935. MAPREDUCE-4741. WARN and ERROR messages logged during normal AM shutdown.
  1936. (Vinod Kumar Vavilapalli via jlowe)
  1937. MAPREDUCE-4730. Fix Reducer's EventFetcher to scale the map-completion
  1938. requests slowly to avoid HADOOP-8942. (Jason Lowe via vinodkv)
  1939. MAPREDUCE-4748. Invalid event: T_ATTEMPT_SUCCEEDED at SUCCEEDED. (jlowe)
  1940. MAPREDUCE-4724. job history web ui applications page should be sorted to
  1941. display last app first (tgraves via bobby)
  1942. MAPREDUCE-4746. The MR Application Master does not have a config to set
  1943. environment variables (Rob Parker via bobby)
  1944. MAPREDUCE-4729. job history UI not showing all job attempts. (Vinod
  1945. Kumar Vavilapalli via jlowe)
  1946. MAPREDUCE-4763 repair test TestUmbilicalProtocolWithJobToken (Ivan A.
  1947. Veselovsky via bobby)
  1948. MAPREDUCE-4771. KeyFieldBasedPartitioner not partitioning properly when
  1949. configured (jlowe via bobby)
  1950. MAPREDUCE-4772. Fetch failures can take way too long for a map to be
  1951. restarted (bobby)
  1952. MAPREDUCE-4782. NLineInputFormat skips first line of last InputSplit
  1953. (Mark Fuhs via bobby)
  1954. MAPREDUCE-4774. JobImpl does not handle asynchronous task events in FAILED
  1955. state (jlowe via bobby)
  1956. MAPREDUCE-4751. AM stuck in KILL_WAIT for days (vinodkv via bobby)
  1957. MAPREDUCE-4787. TestJobMonitorAndPrint is broken (Rob Parker via bobby)
  1958. MAPREDUCE-4425. Speculation + Fetch failures can lead to a hung job (jlowe
  1959. via bobby)
  1960. MAPREDUCE-4786. Job End Notification retry interval is 5 milliseconds by
  1961. default (Ravi Prakash via bobby)
  1962. MAPREDUCE-4517. Too many INFO messages written out during AM to RM heartbeat
  1963. (Jason Lowe via tgraves)
  1964. MAPREDUCE-4797. LocalContainerAllocator can loop forever trying to contact
  1965. the RM (jlowe via bobby)
  1966. MAPREDUCE-4801. ShuffleHandler can generate large logs due to prematurely
  1967. closed channels (jlowe via bobby)
  1968. Release 0.23.4
  1969. INCOMPATIBLE CHANGES
  1970. NEW FEATURES
  1971. IMPROVEMENTS
  1972. MAPREDUCE-2786. Add compression option for TestDFSIO.
  1973. (Plamen Jeliazkov via shv)
  1974. MAPREDUCE-4645. Provide a random seed to Slive to make the sequence
  1975. of file names deterministic. (Ravi Prakash via shv)
  1976. MAPREDUCE-4651. Benchmarking random reads with DFSIO. (shv)
  1977. OPTIMIZATIONS
  1978. BUG FIXES
  1979. MAPREDUCE-4647. We should only unjar jobjar if there is a lib directory
  1980. in it. (Robert Evans via tgraves)
  1981. MAPREDUCE-4691. Historyserver can report "Unknown job" after RM says job
  1982. has completed (Robert Joseph Evans via jlowe)
  1983. MAPREDUCE-4689. JobClient.getMapTaskReports on failed job results in NPE
  1984. (jlowe via bobby)
  1985. Release 0.23.3
  1986. INCOMPATIBLE CHANGES
  1987. MAPREDUCE-4072. User set java.library.path seems to overwrite default
  1988. creating problems native lib loading (Anupam Seth via bobby)
  1989. MAPREDUCE-3812. Lower default allocation sizes, fix allocation
  1990. configurations and document them (Harsh J via bobby)
  1991. NEW FEATURES
  1992. IMPROVEMENTS
  1993. MAPREDUCE-4059. The history server should have a separate pluggable
  1994. storage/query interface. (Robert Evans via tgraves)
  1995. MAPREDUCE-3942. Randomize master key generation for
  1996. ApplicationTokenSecretManager and roll it every so often. (Vinod Kumar
  1997. Vavilapalli via sseth)
  1998. MAPREDUCE-4151. RM scheduler web page should filter apps to those that
  1999. are relevant to scheduling (Jason Lowe via tgraves)
  2000. MAPREDUCE-4134. Remove references of mapred.child.ulimit etc. since they
  2001. are not being used any more (Ravi Prakash via bobby)
  2002. MAPREDUCE-3972. Fix locking and exception issues in JobHistory server.
  2003. (Robert Joseph Evans via sseth)
  2004. MAPREDUCE-4161. create sockets consistently (Daryn Sharp via bobby)
  2005. MAPREDUCE-4079. Allow MR AppMaster to limit ephemeral port range.
  2006. (bobby via tgraves)
  2007. MAPREDUCE-4210. Expose listener address for WebApp (Daryn Sharp via bobby)
  2008. MAPREDUCE-4162. Correctly set token service (Daryn Sharp via bobby)
  2009. MAPREDUCE-4301. Dedupe some strings in MRAM for memory savings
  2010. (bobby via tgraves)
  2011. MAPREDUCE-4267. mavenize pipes (tgraves via bobby)
  2012. MAPREDUCE-4375. Show Configuration Tracability in MR UI (bobby
  2013. via tgraves)
  2014. MAPREDUCE-4569. Fixed TestHsWebServicesJobsQuery to pass on JDK7 by not
  2015. depending on test order. (Thomas Graves via vinodkv)
  2016. OPTIMIZATIONS
  2017. MAPREDUCE-3850. Avoid redundant calls for tokens in TokenCache (Daryn
  2018. Sharp via bobby)
  2019. BUG FIXES
  2020. MAPREDUCE-4092. commitJob Exception does not fail job (Jon Eagles via
  2021. bobby)
  2022. MAPREDUCE-4089. Hung Tasks never time out. (Robert Evans via tgraves)
  2023. MAPREDUCE-4024. RM webservices can't query on finalStatus (Tom Graves
  2024. via bobby)
  2025. MAPREDUCE-4060. Multiple SLF4J binding warning (Jason Lowe via bobby)
  2026. MAPREDUCE-3983. TestTTResourceReporting can fail, and should just be
  2027. deleted (Ravi Prakash via bobby)
  2028. MAPREDUCE-4012 Hadoop Job setup error leaves no useful info to users
  2029. (when LinuxTaskController is used). (tgraves)
  2030. MAPREDUCE-4062. AM Launcher thread can hang forever (tgraves via bobby)
  2031. MAPREDUCE-3988. mapreduce.job.local.dir doesn't point to a single directory
  2032. on a node. (Eric Payne via bobby)
  2033. MAPREDUCE-3999. Tracking link gives an error if the AppMaster hasn't
  2034. started yet (Ravi Prakash via bobby)
  2035. MAPREDUCE-4020. Web services returns incorrect JSON for deep queue tree
  2036. (Anupam Seth via tgraves)
  2037. MAPREDUCE-3672. Killed maps shouldn't be counted towards
  2038. JobCounter.NUM_FAILED_MAPS. (Anupam Seth via tgraves)
  2039. MAPREDUCE-3682 Tracker URL says AM tasks run on localhost.
  2040. (Ravi Prakash via tgraves)
  2041. MAPREDUCE-3082. Archive command take wrong path for input file with current
  2042. directory (John George via bobby)
  2043. MAPREDUCE-3650. testGetTokensForHftpFS() fails (Ravi Prakash via bobby)
  2044. MAPREDUCE-3621. TestDBJob and TestDataDrivenDBInputFormat ant tests fail
  2045. (Ravi Prakash via tgraves)
  2046. MAPREDUCE-4073. CS assigns multiple off-switch containers when using
  2047. multi-level-queues (Siddharth Seth via bobby)
  2048. MAPREDUCE-4051. Remove the empty hadoop-mapreduce-project/assembly/all.xml
  2049. file (Ravi Prakash via bobby)
  2050. MAPREDUCE-4117. mapred job -status throws NullPointerException (Devaraj K
  2051. via bobby)
  2052. MAPREDUCE-4099. ApplicationMaster may fail to remove staging directory
  2053. (Jason Lowe via bobby)
  2054. MAPREDUCE-4017. Add jobname to jobsummary log (tgraves and Koji Noguchi
  2055. via bobby)
  2056. MAPREDUCE-4040. History links should use hostname rather than IP address.
  2057. (Bhallamudi Venkata Siva Kamesh via sseth)
  2058. MAPREDUCE-4099 amendment. ApplicationMaster will remove staging directory
  2059. after the history service is stopped. (Jason Lowe via sseth)
  2060. MAPREDUCE-3932. Fix the TaskAttempt state machine to handle
  2061. CONTIANER_LAUNCHED and CONTIANER_LAUNCH_FAILED events in additional
  2062. states. (Robert Joseph Evans via sseth)
  2063. MAPREDUCE-4140. mapreduce classes incorrectly importing
  2064. "clover.org.apache.*" classes. (Patrick Hunt via tomwhite)
  2065. MAPREDUCE-4050. For tasks without assigned containers, changes the node
  2066. text on the UI to N/A instead of a link to null. (Bhallamudi Venkata Siva
  2067. Kamesh via sseth)
  2068. MAPREDUCE-4128. AM Recovery expects all attempts of a completed task to
  2069. also be completed. (Bikas Saha via bobby)
  2070. MAPREDUCE-4144. Fix a NPE in the ResourceManager when handling node
  2071. updates. (Jason Lowe via sseth)
  2072. MAPREDUCE-4156. ant build fails compiling JobInProgress (tgraves)
  2073. MAPREDUCE-4160. some mrv1 ant tests fail with timeout - due to 4156
  2074. (tgraves)
  2075. MAPREDUCE-4074. Client continuously retries to RM When RM goes down
  2076. before launching Application Master (xieguiming via tgraves)
  2077. MAPREDUCE-4159. Job is running in Uber mode after setting
  2078. "mapreduce.job.ubertask.maxreduces" to zero (Devaraj K via bobby)
  2079. MAPREDUCE-4165. Committing is misspelled as commiting in task logs
  2080. (John Eagles via bobby)
  2081. MAPREDUCE-4129. Lots of unneeded counters log messages (Ahmed Radwan via
  2082. bobby)
  2083. MAPREDUCE-3947. yarn.app.mapreduce.am.resource.mb not documented
  2084. (Devaraj K via bobby)
  2085. MAPREDUCE-4190. Improve web UI for task attempts userlog link (Tom Graves
  2086. via bobby)
  2087. MAPREDUCE-4133. MR over viewfs is broken (John George via bobby)
  2088. MAPREDUCE-4194. ConcurrentModificationError in DirectoryCollection
  2089. (Jonathan Eagles via bobby)
  2090. MAPREDUCE-3613. web service calls header contains 2 content types
  2091. (tgraves)
  2092. MAPREDUCE-4169. Container Logs appear in unsorted order (Jonathan Eagles
  2093. via bobby)
  2094. MAPREDUCE-4189. TestContainerManagerSecurity is failing (Devaraj K via
  2095. bobby)
  2096. MAPREDUCE-4209. junit dependency in hadoop-mapreduce-client is missing
  2097. scope test (Radim Kolar via bobby)
  2098. MAPREDUCE-4206. Sorting by Last Health-Update on the RM nodes page sorts
  2099. does not work correctly (Jonathon Eagles via tgraves)
  2100. MAPREDUCE-4212. TestJobClientGetJob sometimes fails
  2101. (Daryn Sharp via tgraves)
  2102. MAPREDUCE-4211. Error conditions (missing appid, appid not found) are
  2103. masked in the RM app page (Jonathan Eagles via bobby)
  2104. MAPREDUCE-4163. consistently set the bind address (Daryn Sharp via bobby)
  2105. MAPREDUCE-4048. NullPointerException exception while accessing the
  2106. Application Master UI (Devaraj K via bobby)
  2107. MAPREDUCE-4220. RM apps page starttime/endtime sorts are incorrect
  2108. (Jonathan Eagles via bobby)
  2109. MAPREDUCE-4226. ConcurrentModificationException in FileSystemCounterGroup.
  2110. (tomwhite)
  2111. MAPREDUCE-4215. RM app page shows 500 error on appid parse error
  2112. (Jonathon Eagles via tgraves)
  2113. MAPREDUCE-4237. TestNodeStatusUpdater can fail if localhost has a domain
  2114. associated with it (bobby)
  2115. MAPREDUCE-4233. NPE can happen in RMNMNodeInfo. (bobby)
  2116. MAPREDUCE-4238. mavenize data_join. (tgraves)
  2117. MAPREDUCE-4102. job counters not available in Jobhistory webui for
  2118. killed jobs (Bhallamudi Venkata Siva Kamesh via tgraves)
  2119. MAPREDUCE-3543. Mavenize Gridmix. (tgraves)
  2120. MAPREDUCE-4197. Include the hsqldb jar in the hadoop-mapreduce tar
  2121. file (Ravi Prakash via tgraves)
  2122. MAPREDUCE-4269. documentation: Gridmix has javadoc warnings in
  2123. StressJobFactory (Jonathon Eagles via tgraves).
  2124. MAPREDUCE-3870. Invalid App Metrics
  2125. (Bhallamudi Venkata Siva Kamesh via tgraves).
  2126. MAPREDUCE-4152. map task left hanging after AM dies trying to connect to RM
  2127. (Tom Graves via bobby)
  2128. MAPREDUCE-4297. Usersmap file in gridmix should not fail on empty lines
  2129. (Ravi Prakash via bobby)
  2130. MAPREDUCE-4302. NM goes down if error encountered during log aggregation
  2131. (Daryn Sharp via bobby)
  2132. MAPREDUCE-3350. Per-app RM page should have the list of application-attempts
  2133. like on the app JHS page (Jonathon Eagles via tgraves)
  2134. MAPREDUCE-3842. Stop webpages from automatic refreshing (tgraves)
  2135. MAPREDUCE-3927. Shuffle hang when set map.failures.percent
  2136. (Bhallamudi Venkata Siva Kamesh via tgraves)
  2137. MAPREDUCE-4311. Capacity scheduler.xml does not accept decimal values for
  2138. capacity and maximum-capacity settings (Karthik Kambatla via tgraves)
  2139. MAPREDUCE-4341. add types to capacity scheduler properties documentation
  2140. (Karthik Kambatla via tgraves)
  2141. MAPREDUCE-4270. Move the data_join test classes to the correct path.
  2142. (Thomas Graves via sseth)
  2143. MAPREDUCE-3889. job client tries to use /tasklog interface, but that
  2144. doesn't exist anymore (Devaraj K via bobby)
  2145. MAPREDUCE-4320. gridmix mainClass wrong in pom.xml (tgraves)
  2146. MAPREDUCE-4295. RM crashes due to DNS issue (tgraves)
  2147. MAPREDUCE-4228. mapreduce.job.reduce.slowstart.completedmaps is not working
  2148. properly (Jason Lowe via bobby)
  2149. MAPREDUCE-4392. Counters.makeCompactString() changed behavior from 0.20
  2150. (Jason Lowe via bobby)
  2151. MAPREDUCE-4384. Race conditions in IndexCache (Kihwal Lee via tgraves)
  2152. MAPREDUCE-4387. RM gets fatal error and exits during TestRM
  2153. (Kihwal Lee via tgraves)
  2154. MAPREDUCE-4379. Node Manager throws java.lang.OutOfMemoryError: Java heap
  2155. space due to org.apache.hadoop.fs.LocalDirAllocator.contexts (Devaraj K
  2156. via bobby)
  2157. MAPREDUCE-4402. TestFileInputFormat fails intermittently (Jason Lowe via
  2158. bobby)
  2159. MAPREDUCE-4300. OOM in AM can turn it into a zombie. (Robert Evans via
  2160. tgraves)
  2161. MAPREDUCE-4252. MR2 job never completes with 1 pending task (Tom White via
  2162. bobby)
  2163. MAPREDUCE-3940. ContainerTokens should have an expiry interval. (Siddharth
  2164. Seth and Vinod Kumar Vavilapalli via vinodkv)
  2165. MAPREDUCE-4419. ./mapred queue -info <queuename> -showJobs displays all
  2166. the jobs irrespective of <queuename> (Devaraj K via bobby)
  2167. MAPREDUCE-4299. Terasort hangs with MR2 FifoScheduler (Tom White via
  2168. bobby)
  2169. MAPREDUCE-4437. Race in MR ApplicationMaster can cause reducers to never be
  2170. scheduled (Jason Lowe via bobby)
  2171. MAPREDUCE-4449. Incorrect MR_HISTORY_STORAGE property name in JHAdminConfig
  2172. (Ahmed Radwan via bobby)
  2173. MAPREDUCE-4283. Display tail of aggregated logs by default (Jason Lowe via
  2174. bobby)
  2175. MAPREDUCE-4448. Fix NM crash during app cleanup if aggregation didn't
  2176. init. (Jason Lowe via daryn)
  2177. MAPREDUCE-3893. allow capacity scheduler configs maximum-applications and
  2178. maximum-am-resource-percent configurable on a per queue basis (tgraves via
  2179. bobby)
  2180. MAPREDUCE-4467. IndexCache failures due to missing synchronization
  2181. (Kihwal Lee via tgraves)
  2182. MAPREDUCE-4423. Potential infinite fetching of map output (Robert Evans
  2183. via tgraves)
  2184. MAPREDUCE-4456. LocalDistributedCacheManager can get an
  2185. ArrayIndexOutOfBounds when creating symlinks (Robert Evans via tgraves)
  2186. MAPREDUCE-4496. AM logs link is missing user name (Jason Lowe via bobby)
  2187. MAPREDUCE-4493. Distibuted Cache Compatability Issues (Robert Evans
  2188. via tgraves)
  2189. MAPREDUCE-4492. Configuring total queue capacity between 100.5 and 99.5 at
  2190. perticular level is sucessfull (Mayank Bansal via bobby)
  2191. MAPREDUCE-4457. mr job invalid transition TA_TOO_MANY_FETCH_FAILURE at
  2192. FAILED (Robert Evans via tgraves)
  2193. MAPREDUCE-4234. SortValidator.java is incompatible with multi-user or
  2194. parallel use (due to a /tmp file with static name) (Robert Evans via
  2195. jeagles)
  2196. MAPREDUCE-4504. SortValidator writes to wrong directory (Robert Evans
  2197. via tgraves)
  2198. MAPREDUCE-4503. Should throw InvalidJobConfException if duplicates found in
  2199. cacheArchives or cacheFiles (Robert Evans via jeagles)
  2200. MAPREDUCE-3782. teragen terasort jobs fail when using webhdfs:// (Jason
  2201. Lowe via bobby)
  2202. MAPREDUCE-4053. Counters group names deprecation is wrong, iterating over
  2203. group names deprecated names don't show up (Robert Evans via tgraves)
  2204. MAPREDUCE-3506. Calling getPriority on JobInfo after parsing a history log
  2205. with JobHistoryParser throws a NullPointerException (Jason Lowe via bobby)
  2206. MAPREDUCE-4570. ProcfsBasedProcessTree#constructProcessInfo() prints a
  2207. warning if procfsDir/<pid>/stat is not found. (Ahmed Radwan via bobby)
  2208. MAPREDUCE-4600. TestTokenCache.java from MRV1 no longer compiles (daryn
  2209. via bobby)
  2210. MAPREDUCE-4612. job summary file permissions not set when its created
  2211. (tgraves via bobby)
  2212. MAPREDUCE-4614. Simplify debugging a job's tokens (daryn via bobby)
  2213. MAPREDUCE-4611. MR AM dies badly when Node is decommissioned (Robert
  2214. Evans via tgraves)
  2215. MAPREDUCE-4604. In mapred-default, mapreduce.map.maxattempts &
  2216. mapreduce.reduce.maxattempts defaults are set to 4 as well as
  2217. mapreduce.job.maxtaskfailures.per.tracker. (Ravi Prakash via jeagles)
  2218. MAPREDUCE-4633. history server doesn't set permissions on all subdirs
  2219. (tgraves via bobby)
  2220. MAPREDUCE-4641. Exception in commitJob marks job as successful in job
  2221. history (Jason Lowe via bobby)
  2222. MAPREDUCE-4549. Distributed cache conflicts breaks backwards compatability
  2223. (Robert Evans via tucu)
  2224. Release 0.23.2 - UNRELEASED
  2225. INCOMPATIBLE CHANGES
  2226. NEW FEATURES
  2227. IMPROVEMENTS
  2228. MAPREDUCE-3849. Change TokenCache's reading of the binary token file
  2229. (Daryn Sharp via bobby)
  2230. MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's
  2231. environmental variables in TestMiniMRChildTask. (Tom White via vinodkv)
  2232. MAPREDUCE-3877 Add a test to formalise the current state transitions
  2233. of the yarn lifecycle. (stevel)
  2234. MAPREDUCE-3866. Fixed the bin/yarn script to not print the command line
  2235. unnecessarily. (vinodkv)
  2236. MAPREDUCE-3730. Modified RM to allow restarted NMs to be able to join the
  2237. cluster without waiting for expiry. (Jason Lowe via vinodkv)
  2238. MAPREDUCE-2793. Corrected AppIDs, JobIDs, TaskAttemptIDs to be of correct
  2239. format on the web pages. (Bikas Saha via vinodkv)
  2240. MAPREDUCE-3614. Fixed MR AM to close history file quickly and send a correct
  2241. final state to the RM when it is killed. (Ravi Prakash via vinodkv)
  2242. MAPREDUCE-3497. Added docs for YARN CLI. (tgraves via acmurthy)
  2243. MAPREDUCE-3954. Added new envs to separate heap size for different daemons
  2244. started via bin scripts. (Robert Joseph Evans via vinodkv)
  2245. MAPREDUCE-4025. AM can crash if task attempt reports bogus progress value
  2246. (Jason Lowe via bobby)
  2247. MAPREDUCE-4034. Unable to view task logs on history server with
  2248. mapreduce.job.acl-view-job=* (Jason Lowe and Siddarth Seth via bobby)
  2249. MAPREDUCE-4043. Secret keys set in Credentials are not seen by tasks
  2250. (Jason Lowe via bobby)
  2251. MAPREDUCE-3989. Cap space usage of default log4j rolling policy.
  2252. (Patrick Hunt via eli)
  2253. OPTIMIZATIONS
  2254. MAPREDUCE-3901. Modified JobHistory records in YARN to lazily load job and
  2255. task reports so as to improve UI response times. (Siddarth Seth via vinodkv)
  2256. MAPREDUCE-2855. Passing a cached class-loader to ResourceBundle creator to
  2257. minimize counter names lookup time. (Siddarth Seth via vinodkv)
  2258. MAPREDUCE-3944. Change the history jobs/ webservice to return partial job
  2259. info for a significant performance improvement. (Robert Joseph Evans via
  2260. sseth)
  2261. BUG FIXES
  2262. MAPREDUCE-3918 proc_historyserver no longer in command line arguments for
  2263. HistoryServer (Jon Eagles via bobby)
  2264. MAPREDUCE-3862. Nodemanager can appear to hang on shutdown due to lingering
  2265. DeletionService threads (Jason Lowe via bobby)
  2266. MAPREDUCE-3680. FifoScheduler web service rest API can print out invalid
  2267. JSON. (B Anil Kumar via tgraves)
  2268. MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas
  2269. Graves via mahadev)
  2270. MAPREDUCE-3864. Fix cluster setup docs for correct SecondaryNameNode
  2271. HTTPS parameters. (todd)
  2272. MAPREDUCE-3583. Change pid to String and stime to BigInteger in order to
  2273. avoid NumberFormatException caused by overflow. (Zhihong Yu via szetszwo)
  2274. MAPREDUCE-3634. Fixed all daemons to crash instead of hanging around when
  2275. their EventHandlers get exceptions. (vinodkv)
  2276. MAPREDUCE-3798. Fixed failing TestJobCleanup.testCusomCleanup() and moved it
  2277. to the maven build. (Ravi Prakash via vinodkv)
  2278. MAPREDUCE-3884. PWD should be first in the classpath of MR tasks (tucu)
  2279. MAPREDUCE-3878. Null user on filtered jobhistory job page (Jonathon Eagles
  2280. via tgraves)
  2281. MAPREDUCE-3738. MM can hang during shutdown if AppLogAggregatorImpl thread
  2282. dies unexpectedly (Jason Lowe via sseth)
  2283. MAPREDUCE-3904 Job history produced with mapreduce.cluster.acls.enabled
  2284. false can not be viewed with mapreduce.cluster.acls.enabled true
  2285. (Jonathon Eagles via tgraves)
  2286. MAPREDUCE-3910. Fixed a bug in CapacityScheduler LeafQueue which was causing
  2287. app-submission to fail. (John George via vinodkv)
  2288. MAPREDUCE-3686. Fixed two bugs in Counters because of which web app displays
  2289. zero counter values for framework counters. (Bhallamudi Venkata Siva Kamesh
  2290. via vinodkv)
  2291. MAPREDUCE-3913. RM application webpage is unresponsive after 2000 jobs
  2292. (Jason Lowe via tgraves)
  2293. MAPREDUCE-3922. Fixed build to not compile 32bit container-executor binary
  2294. by default on all platforms. (Hitesh Shah via vinodkv)
  2295. MAPREDUCE-3790. Broken pipe on streaming job can lead to truncated output for
  2296. a successful job (Jason Lowe via bobby)
  2297. MAPREDUCE-3816. capacity scheduler web ui bar graphs for used capacity wrong
  2298. (tgraves via bobby)
  2299. MAPREDUCE-3930. Fixed an NPE while accessing the AM page/webservice for a
  2300. task attempt without an assigned container. (Robert Joseph Evans via
  2301. sseth)
  2302. MAPREDUCE-3931. Changed PB implementation of LocalResource to take locks
  2303. so that race conditions don't fail tasks by inadvertantly changing the
  2304. timestamps. (Siddarth Seth via vinodkv)
  2305. MAPREDUCE-3687. If AM dies before it returns new tracking URL, proxy
  2306. redirects to http://N/A/ and doesn't return error code (Ravi Prakash via
  2307. bobby)
  2308. MAPREDUCE-3920. Revise yarn default port number selection
  2309. (Dave Thompson via tgraves)
  2310. MAPREDUCE-3903. Add support for mapreduce admin users. (Thomas Graves via
  2311. sseth)
  2312. MAPREDUCE-3706. Fix circular redirect error in job-attempts page. (bobby
  2313. via acmurthy)
  2314. MAPREDUCE-3896. Add user information to the delegation token issued by the
  2315. history server. (Vinod Kumar Vavilapalli via sseth)
  2316. MAPREDUCE-3792. Fix "bin/mapred job -list" to display all jobs instead of
  2317. only the jobs owned by the user. (Jason Lowe via vinodkv)
  2318. MAPREDUCE-3929. Fixed output of 'bin/mapred queue -showacl' command to
  2319. clarify ACLs for users. (John George via acmurthy)
  2320. MAPREDUCE-3960. Fix web-proxy to forward request to AM with configured
  2321. hostname or IP. (tgraves via acmurthy)
  2322. MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by
  2323. using capacity and not max-capacity since we are already scaling it by
  2324. userLimitFactor. (Eric Payne via acmurthy)
  2325. MAPREDUCE-3009. Fixed node link on JobHistory webapp. (chackaravarthy via
  2326. vinodkv)
  2327. MAPREDUCE-3964. ResourceManager does not have JVM metrics (Jason Lowe via
  2328. bobby)
  2329. MAPREDUCE-3034. Ensure NodeManager reboots itself on direction from
  2330. ResourceManager. (Devaraj K & Eric Payne via acmurthy)
  2331. MAPREDUCE-3976. TestRMContainerAllocator failing (Jason Lowe via bobby)
  2332. MAPREDUCE-3961. Map/ReduceSlotMillis computation incorrect (Siddharth Seth
  2333. via bobby)
  2334. MAPREDUCE-3977. LogAggregationService leaks log aggregator objects
  2335. (Jason Lowe via bobby)
  2336. MAPREDUCE-3975. Default value not set for Configuration parameter
  2337. mapreduce.job.local.dir (Eric Payne via bobby)
  2338. MAPREDUCE-3982. Fixed FileOutputCommitter to not err out for an 'empty-job'
  2339. whose tasks don't write any outputs. (Robert Joseph Evans via vinodkv)
  2340. MAPREDUCE-4005. Fixes broken AM container logs URL on ResourceManager
  2341. Application Info page. (Jason Lowe via sseth)
  2342. MAPREDUCE-4006. History server container log web UI sometimes combines
  2343. stderr/stdout/syslog contents together (Siddharth Seth via tgraves)
  2344. MAPREDUCE-4061. RM only has 1 AM launcher thread (tgraves via bobby)
  2345. Release 0.23.1 - 2012-02-17
  2346. NEW FEATURES
  2347. MAPREDUCE-778. Rumen Anonymizer. (Amar Kamat and Chris Douglas via amarrk)
  2348. MAPREDUCE-3121. NodeManager should handle disk-failures (Ravi Gummadi via mahadev)
  2349. MAPREDUCE-2863. Support web services for YARN and MR components. (Thomas
  2350. Graves via vinodkv)
  2351. MAPREDUCE-3251. Network ACLs can prevent some clients to talk to MR ApplicationMaster.
  2352. (Anupam Seth via mahadev)
  2353. MAPREDUCE-778. Rumen Anonymizer. (Amar Kamat and Chris Douglas via amarrk)
  2354. IMPROVEMENTS
  2355. MAPREDUCE-3481. [Gridmix] Improve Gridmix STRESS mode. (amarrk)
  2356. MAPREDUCE-3597. [Rumen] Rumen should provide APIs to access all the
  2357. job-history related information.
  2358. MAPREDUCE-3375. [Gridmix] Memory Emulation system tests.
  2359. (Vinay Thota via amarrk)
  2360. MAPREDUCE-3840. JobEndNotifier doesn't use the proxyToUse during connecting
  2361. (Ravi Prakash via bobby)
  2362. MAPREDUCE-3736. Variable substitution depth too large for fs.default.name
  2363. causes jobs to fail (ahmed via tucu).
  2364. MAPREDUCE-2733. [Gridmix] Gridmix3 cpu emulation system tests.
  2365. (Vinay Thota via amarrk)
  2366. MAPREDUCE-3297. Moved log related components into yarn-common so that
  2367. HistoryServer and clients can use them without depending on the
  2368. yarn-server-nodemanager module. (Siddharth Seth via vinodkv)
  2369. MAPREDUCE-3336. Replaced guice internal.Preconditions api usage with the
  2370. public Preconditions API. (Thomas Graves via vinodkv)
  2371. MAPREDUCE-3280. Removed the unnecessary job user-name configuration in
  2372. mapred-site.xml. (vinodkv)
  2373. MAPREDUCE-3370. Fixed MiniMRYarnCluster and related tests to not use
  2374. a hard-coded path for the mr-app jar. (Ahmed Radwan via vinodkv)
  2375. MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves
  2376. via mahadev)
  2377. MAPREDUCE-3341. Enhance logging of initalized queue limit values.
  2378. (Anupam Seth via mahadev)
  2379. MAPREDUCE-3243. Invalid tracking URL for streaming jobs (Jonathan Eagles
  2380. via mahadev)
  2381. MAPREDUCE-3331. Improvement to single node cluster setup documentation for
  2382. 0.23 (Anupam Seth via mahadev)
  2383. MAPREDUCE-3102. Changed NodeManager to fail fast when LinuxContainerExecutor
  2384. has wrong configuration or permissions. (Hitesh Shah via vinodkv)
  2385. MAPREDUCE-3415. improve MiniMRYarnCluster & DistributedShell JAR
  2386. resolution. (tucu)
  2387. MAPREDUCE-3169. Create a new MiniMRCluster equivalent which only provides
  2388. client APIs cross MR1 and MR2. (Ahmed via tucu)
  2389. MAPREDUCE-3373. Hadoop scripts unconditionally source
  2390. "$bin"/../libexec/hadoop-config.sh. (Bruno Mahé via tomwhite)
  2391. MAPREDUCE-3372. HADOOP_PREFIX cannot be overridden.
  2392. (Bruno Mahé via tomwhite)
  2393. MAPREDUCE-3411. Performance Upgrade for jQuery (Jonathan Eagles via
  2394. mahadev)
  2395. MAPREDUCE-3371. Review and improve the yarn-api javadocs. (Ravi Prakash
  2396. via mahadev)
  2397. MAPREDUCE-3238. Small cleanup in SchedulerApp. (Todd Lipcon via mahadev)
  2398. MAPREDUCE-3413. RM web ui applications not sorted in any order by default.
  2399. (Jonathan Eagles via mahadev)
  2400. MAPREDUCE-3045. Fixed UI filters to not filter on hidden title-numeric
  2401. sort fields. (Jonathan Eagles via sseth)
  2402. MAPREDUCE-3448. TestCombineOutputCollector javac unchecked warning on mocked
  2403. generics (Jonathan Eagles via mahadev)
  2404. MAPREDUCE-3169 amendment. Deprecate MiniMRCluster. (Ahmed Radwan via
  2405. sseth)
  2406. MAPREDUCE-3369. Migrate MR1 tests to run on MR2 using the new interfaces
  2407. introduced in MAPREDUCE-3169. (Ahmed Radwan via tomwhite)
  2408. MAPREDUCE-3518. mapred queue -info <queue> -showJobs throws NPE.
  2409. (Jonathan Eagles via mahadev)
  2410. MAPREDUCE-3391. Making a trivial change to correct a log message in
  2411. DistributedShell app's AM. (Subroto Sanyal via vinodkv)
  2412. MAPREDUCE-3547. Added a bunch of unit tests for the the RM/NM webservices.
  2413. (Thomas Graves via acmurthy)
  2414. MAPREDUCE-3610. Remove use of the 'dfs.block.size' config for default block
  2415. size fetching. Use FS#getDefaultBlocksize instead. (Sho Shimauchi via harsh)
  2416. MAPREDUCE-3478. Cannot build against ZooKeeper 3.4.0. (Tom White via mahadev)
  2417. MAPREDUCE-3528. Fixed TaskHeartBeatHandler to use a new configuration
  2418. for the thread loop interval separate from task-timeout configuration
  2419. property. (Siddharth Seth via vinodkv)
  2420. MAPREDUCE-3312. Modified MR AM to not send a stop-container request for
  2421. a container that isn't launched at all. (Robert Joseph Evans via vinodkv)
  2422. MAPREDUCE-3382. Enhanced MR AM to use a proxy to ping the job-end
  2423. notification URL. (Ravi Prakash via vinodkv)
  2424. MAPREDUCE-3299. Added AMInfo table to the MR AM job pages to list all the
  2425. job-attempts when AM restarts and recovers. (Jonathan Eagles via vinodkv)
  2426. MAPREDUCE-3251. Network ACLs can prevent some clients to talk to MR AM.
  2427. Improved the earlier patch to not to JobHistoryServer repeatedly.
  2428. (Anupam Seth via vinodkv)
  2429. MAPREDUCE-3553. Add support for data returned when exceptions thrown from web
  2430. service apis to be in either xml or in JSON. (Thomas Graves via mahadev)
  2431. MAPREDUCE-3641. Making CapacityScheduler more conservative so as to
  2432. assign only one off-switch container in a single scheduling
  2433. iteration. (Arun C Murthy via vinodkv)
  2434. MAPREDUCE-3692. yarn-resourcemanager out and log files can get big. (eli)
  2435. MAPREDUCE-3710. Improved FileInputFormat to return better locality for the
  2436. last split. (Siddarth Seth via vinodkv)
  2437. MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)
  2438. MAPREDUCE-3737. The Web Application Proxy's is not documented very well.
  2439. (Robert Evans via mahadev)
  2440. MAPREDUCE-3699. Increased RPC handlers for all YARN servers to reasonable
  2441. values for working at scale. (Hitesh Shah via vinodkv)
  2442. MAPREDUCE-3693. Added mapreduce.admin.user.env to mapred-default.xml.
  2443. (Roman Shapshonik via acmurthy)
  2444. MAPREDUCE-3732. Modified CapacityScheduler to use only users with pending
  2445. requests for computing user-limits. (Arun C Murthy via vinodkv)
  2446. MAPREDUCE-3679. AM logs and others should not automatically refresh after every 1
  2447. second. (Vinod KV via mahadev)
  2448. MAPREDUCE-3754. Modified RM UI to filter applications based on state of the
  2449. applications. (vinodkv)
  2450. MAPREDUCE-3774. Moved yarn-default.xml to hadoop-yarn-common from
  2451. hadoop-server-common. (Mahadev Konar via vinodkv)
  2452. MAPREDUCE-3771. Un-deprecated the old mapred apis, port of MAPREDUCE-1735.
  2453. (acmurthy)
  2454. MAPREDUCE-3784. Fixed CapacityScheduler so that maxActiveApplications and
  2455. maxActiveApplicationsPerUser per queue are not too low for small
  2456. clusters. (Arun C Murthy via vinodkv)
  2457. OPTIMIZATIONS
  2458. MAPREDUCE-3567. Extraneous JobConf objects in AM heap. (Vinod Kumar
  2459. Vavilapalli via sseth)
  2460. MAPREDUCE-3399. Modifying ContainerLocalizer to send a heartbeat to NM
  2461. immediately after downloading a resource instead of always waiting for a
  2462. second. (Siddarth Seth via vinodkv)
  2463. MAPREDUCE-3568. Optimized Job's progress calculations in MR AM. (vinodkv)
  2464. MAPREDUCE-3569. TaskAttemptListener holds a global lock for all
  2465. task-updates. (Vinod Kumar Vavilapalli via sseth)
  2466. MAPREDUCE-3511. Removed a multitude of cloned/duplicate counters in the AM
  2467. thereby reducing the AM heap size and preventing full GCs. (vinodkv)
  2468. MAPREDUCE-3618. Fixed TaskHeartbeatHandler to not hold a global lock for all
  2469. task-updates. (Siddarth Seth via vinodkv)
  2470. MAPREDUCE-3512. Batching JobHistory flushing to DFS so that we don't flush
  2471. for every event slowing down AM. (Siddarth Seth via vinodkv)
  2472. MAPREDUCE-3718. Change default AM heartbeat interval to 1 second. (Hitesh
  2473. Shah via sseth)
  2474. MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes
  2475. on the webapps. (Bhallamudi Venkata Siva Kamesh and Jason Lowe via vinodkv)
  2476. MAPREDUCE-3756. Made single shuffle limit configurable. (Hitesh Shah via
  2477. acmurthy)
  2478. MAPREDUCE-3811. Made jobclient-to-AM retries configurable. (sseth via
  2479. acmurthy)
  2480. BUG FIXES
  2481. MAPREDUCE-2784. [Gridmix] Bug fixes in ExecutionSummarizer and
  2482. ResourceUsageMatcher. (amarrk)
  2483. MAPREDUCE-3194. "mapred mradmin" command is broken in mrv2
  2484. (Jason Lowe via bobby)
  2485. MAPREDUCE-3462. Fix Gridmix JUnit testcase failures.
  2486. (Ravi Prakash and Ravi Gummadi via amarrk)
  2487. MAPREDUCE-2950. [Rumen] Fixed TestUserResolve. (Ravi Gummadi via amarrk)
  2488. MAPREDUCE-3412. Fix 'ant docs'. (amarrk)
  2489. MAPREDUCE-3346 [Rumen] LoggedTaskAttempt#getHostName() returns null.
  2490. (amarrk)
  2491. MAPREDUCE-3221. Reenabled the previously ignored test in TestSubmitJob
  2492. and fixed bugs in it. (Devaraj K via vinodkv)
  2493. MAPREDUCE-3215. Reenabled and fixed bugs in the failing test
  2494. TestNoJobSetupCleanup. (Hitesh Shah via vinodkv)
  2495. MAPREDUCE-3219. Reenabled and fixed bugs in the failing test
  2496. TestDelegationToken. (Hitesh Shah via vinodkv)
  2497. MAPREDUCE-3217. Reenabled and fixed bugs in the failing ant test
  2498. TestAuditLogger. (Devaraj K via vinodkv)
  2499. MAPREDUCE-3291. App fail to launch due to delegation token not
  2500. found in cache (Robert Evans via mahadev)
  2501. MAPREDUCE-3344. o.a.h.mapreduce.Reducer since 0.21 blindly casts to
  2502. ReduceContext.ValueIterator. (Brock Noland via tomwhite)
  2503. MAPREDUCE-3342. Fixed JobHistoryServer to also show the job's queue
  2504. name. (Jonathan Eagles via vinodkv)
  2505. MAPREDUCE-3345. Fixed a race condition in ResourceManager that was causing
  2506. TestContainerManagerSecurity to fail sometimes. (Hitesh Shah via vinodkv)
  2507. MAPREDUCE-3368. Fixed test compilation. (Hitesh Shah via vinodkv)
  2508. MAPREDUCE-3333. Fixed bugs in ContainerLauncher of MR AppMaster due to
  2509. which per-container connections to NodeManager were lingering long enough
  2510. to hit the ulimits on number of processes. (vinodkv)
  2511. MAPREDUCE-3392. Fixed Cluster's getDelegationToken's API to return null
  2512. when there isn't a supported token. (John George via vinodkv)
  2513. MAPREDUCE-3379. Fixed LocalResourceTracker in NodeManager to remove deleted
  2514. cache entries correctly. (Siddharth Seth via vinodkv)
  2515. MAPREDUCE-3324. Not All HttpServer tools links (stacks,logs,config,metrics) are
  2516. accessible through all UI servers (Jonathan Eagles via mahadev)
  2517. MAPREDUCE-3355. Fixed MR AM's ContainerLauncher to handle node-command
  2518. timeouts correctly. (vinodkv)
  2519. MAPREDUCE-3407. Fixed pom files to refer to the correct MR app-jar needed
  2520. by the integration tests. (Hitesh Shah via vinodkv)
  2521. MAPREDUCE-3437. Fix examples pom to refer to the correct 0.23 snapshot
  2522. version. (Jonathan Eagles via todd)
  2523. MAPREDUCE-3434. Nightly build broken (Hitesh Shah via mahadev)
  2524. MAPREDUCE-3447. mapreduce examples not working (mahadev)
  2525. MAPREDUCE-3444. trunk/0.23 builds broken (Hitesh Shah via mahadev)
  2526. MAPREDUCE-3454. [Gridmix] TestDistCacheEmulation is broken (Hitesh Shah
  2527. via mahadev)
  2528. MAPREDUCE-3408. yarn-daemon.sh unconditionnaly sets yarn.root.logger
  2529. (Bruno Mahe via mahadev)
  2530. MAPREDUCE-3329. Fixed CapacityScheduler to ensure maximum-capacity cannot
  2531. be lesser than capacity for any queue. (acmurthy)
  2532. MAPREDUCE-3464. mapreduce jsp pages missing DOCTYPE. (Dave Vronay via mattf)
  2533. MAPREDUCE-3265. Removed debug logs during job submission to LOG.debug to
  2534. cut down noise. (acmurthy)
  2535. MAPREDUCE-3468. Changed ant based infrastructure to use 0.23.1 version.
  2536. (sseth via acmurthy)
  2537. MAPREDUCE-3433. Finding counters by legacy group name returns empty
  2538. counters. (tomwhite)
  2539. MAPREDUCE-3450. NM port info no longer available in JobHistory.
  2540. (Siddharth Seth via mahadev)
  2541. MAPREDUCE-3477. Hadoop site documentation cannot be built anymore.
  2542. (jeagles via tucu)
  2543. MAPREDUCE-3488. Streaming jobs are failing because the main class
  2544. isnt set in the pom files. (mahadev)
  2545. MAPREDUCE-3463. Second AM fails to recover properly when first AM is killed with
  2546. java.lang.IllegalArgumentException causing lost job. (Siddharth Seth via mahadev)
  2547. MAPREDUCE-3452. fifoscheduler web ui page always shows 0% used for the queue.
  2548. (Jonathan Eagles via mahadev)
  2549. MAPREDUCE-3443. JobClient and Job should function in the context of the
  2550. UGI which created them. (Mahadev Konar via sseth)
  2551. MAPREDUCE-3460. MR AM can hang if containers are allocated on a node
  2552. blacklisted by the AM. (Hitesh Shah and Robert Joseph Evans via sseth)
  2553. MAPREDUCE-3453. RM web ui application details page shows RM cluster about
  2554. information. (Jonathan Eagles via sseth)
  2555. MAPREDUCE-3479. JobClient#getJob cannot find local jobs. (tomwhite)
  2556. MAPREDUCE-3500. MRJobConfig creates an LD_LIBRARY_PATH using the platform ARCH. (tucu)
  2557. MAPREDUCE-3456. $HADOOP_PREFIX/bin/yarn should set defaults for
  2558. $HADOOP_*_HOME (Eric Payne via mahadev)
  2559. MAPREDUCE-3458. Fix findbugs warnings in hadoop-examples. (Devaraj K
  2560. via mahadev)
  2561. MAPREDUCE-3485. DISKS_FAILED -101 error code should be defined in same location as
  2562. ABORTED_CONTAINER_EXIT_STATUS. (Ravi Gummadi via mahadev)
  2563. MAPREDUCE-3389. MRApps loads the 'mrapp-generated-classpath' file with
  2564. classpath from the build machine. (tucu)
  2565. MAPREDUCE-3496. Fixed client to print queue acls in consistent order.
  2566. (Jonathan Eagles via acmurthy)
  2567. MAPREDUCE-3147. Handle leaf queues with the same name properly.
  2568. (Ravi Prakash via mahadev)
  2569. MAPREDUCE-3327. RM web ui scheduler link doesn't show correct max value
  2570. for queues (Anupam Seth via mahadev)
  2571. MAPREDUCE-3513. Capacity Scheduler web UI has a spelling mistake for Memory.
  2572. (chackaravarthy via mahadev)
  2573. MAPREDUCE-3519. Fixed a deadlock in NodeManager LocalDirectories's handling
  2574. service. (Ravi Gummadi via vinodkv)
  2575. MAPREDUCE-3527. Fix minor API incompatibilities between 1.0 and 0.23.
  2576. (tomwhite)
  2577. MAPREDUCE-3328. mapred queue -list output inconsistent and missing child
  2578. queues. (Ravi Prakash via mahadev)
  2579. MAPREDUCE-3510. Capacity Scheduler inherited ACLs not displayed by mapred queue
  2580. -showacls (Jonathan Eagles via mahadev)
  2581. MAPREDUCE-3537. Fix race condition in DefaultContainerExecutor which led
  2582. to container localization occuring in wrong directories. (acmurthy)
  2583. MAPREDUCE-3542. Support "FileSystemCounter" legacy counter group name for
  2584. compatibility. (tomwhite)
  2585. MAPREDUCE-3426. Fixed MR AM in uber mode to write map intermediate outputs
  2586. in the correct directory to work properly in secure mode. (Hitesh Shah via
  2587. vinodkv)
  2588. MAPREDUCE-3544. gridmix build is broken, requires hadoop-archives to be
  2589. added as ivy dependency. (tucu)
  2590. MAPREDUCE-3557. MR1 test fail to compile because of missing hadoop-archives
  2591. dependency. (tucu)
  2592. MAPREDUCE-3541. Fix broken TestJobQueueClient test. (Ravi Prakash via
  2593. mahadev)
  2594. MAPREDUCE-3398. Fixed log aggregation to work correctly in secure mode.
  2595. (Siddharth Seth via vinodkv)
  2596. MAPREDUCE-3530. Fixed an NPE occuring during scheduling in the
  2597. ResourceManager. (Arun C Murthy via vinodkv)
  2598. MAPREDUCE-3484. Fixed JobEndNotifier to not get interrupted before completing
  2599. all its retries. (Ravi Prakash via vinodkv)
  2600. MAPREDUCE-3531. Fixed a race in ContainerTokenSecretManager. (Robert Joseph
  2601. Evans via sseth)
  2602. MAPREDUCE-3560. TestRMNodeTransitions is failing on trunk.
  2603. (Siddharth Seth via mahadev)
  2604. MAPREDUCE-3487. Fixed JobHistory web-UI to display links to single task's
  2605. counters' page. (Jason Lowe via vinodkv)
  2606. MAPREDUCE-3564. Fixed failures in TestStagingCleanup and TestJobEndNotifier
  2607. tests. (Siddharth Seth via vinodkv)
  2608. MAPREDUCE-3422. Counter display names are not being picked up. (Jonathan
  2609. Eagles via sseth)
  2610. MAPREDUCE-3366. Mapreduce component should use consistent directory structure
  2611. layout as HDFS/common (Eric Yang via mahadev)
  2612. MAPREDUCE-3387. Fixed AM's tracking URL to always go through the proxy, even
  2613. before the job started, so that it works properly with oozie throughout
  2614. the job execution. (Robert Joseph Evans via vinodkv)
  2615. MAPREDUCE-3579. ConverterUtils shouldn't include a port in a path from a url
  2616. without a port. (atm via harsh)
  2617. MAPREDUCE-3563. Fixed LocalJobRunner to work correctly with new mapreduce
  2618. apis. (acmurthy)
  2619. MAPREDUCE-3376. Fixed Task to ensure it passes reporter to combiners using
  2620. old MR api. (Subroto Sanyal via acmurthy)
  2621. MAPREDUCE-3339. Fixed MR AM to stop considering node blacklisting after the
  2622. number of nodes blacklisted crosses a threshold. (Siddharth Seth via vinodkv)
  2623. MAPREDUCE-3588. Fixed bin/yarn which was broken by MAPREDUCE-3366 so that
  2624. yarn daemons can start. (Arun C Murthy via vinodkv)
  2625. MAPREDUCE-3349. Log rack-name in JobHistory for unsuccessful tasks. (Amar
  2626. Kamat and Devaraj K via sseth)
  2627. MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations
  2628. thereby solving race conditions in MR AM shutdown. (vinodkv)
  2629. MAPREDUCE-3604. Fixed streaming to use new mapreduce.framework.name to
  2630. check for local mode. (acmurthy)
  2631. MAPREDUCE-3521. Fixed streaming to ensure it doesn't silently ignore
  2632. unknown arguments. (Robert Evans via acmurthy)
  2633. MAPREDUCE-3522. Ensure queues inherit ACLs from parent if they aren't
  2634. explicitly specified. (Jonathan Eagles via acmurthy)
  2635. MAPREDUCE-3608. Fixed compile issue with MAPREDUCE-3522. (mahadev via
  2636. acmurthy)
  2637. MAPREDUCE-3490. Fixed MapReduce AM to count failed maps also towards Reduce
  2638. ramp up. (Sharad Agarwal and Arun C Murthy via vinodkv)
  2639. MAPREDUCE-3529. TokenCache does not cache viewfs credentials correctly
  2640. (sseth)
  2641. MAPREDUCE-3595. Add missing TestCounters#testCounterValue test from branch
  2642. 1 to 0.23 (Tom White via sseth)
  2643. MAPREDUCE-3566. Fixed MR AM to construct CLC only once across all tasks.
  2644. (vinodkv via acmurthy)
  2645. MAPREDUCE-3572. Moved AM event dispatcher to a separate thread for
  2646. performance reasons. (vinodkv via acmurthy)
  2647. MAPREDUCE-3615. Fix some ant test failures. (Thomas Graves via sseth)
  2648. MAPREDUCE-1744. DistributedCache creates its own FileSytem instance when
  2649. adding a file/archive to the path. (Dick King via tucu)
  2650. MAPREDUCE-3326. Added detailed information about queue's to the
  2651. CapacityScheduler web-ui. (Jason Lowe via acmurthy)
  2652. MAPREDUCE-3548. Added more unit tests for MR AM & JHS web-services.
  2653. (Thomas Graves via acmurthy)
  2654. MAPREDUCE-3617. Removed wrong default value for
  2655. yarn.resourcemanager.principal and yarn.nodemanager.principal. (Jonathan
  2656. Eagles via acmurthy)
  2657. MAPREDUCE-3624. Remove unnecessary dependency on JDK's tools.jar. (mahadev
  2658. via acmurthy)
  2659. MAPREDUCE-3616. Thread pool for launching containers in MR AM not
  2660. expanding as expected. (vinodkv via sseth)
  2661. MAPREDUCE-3639. Fixed TokenCache to work with absent FileSystem canonical
  2662. service-names. (Siddharth Seth via vinodkv)
  2663. MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos
  2664. authenticated. (mahadev)
  2665. MAPREDUCE-3648. TestJobConf failing. (Thomas Graves via mahadev)
  2666. MAPREDUCE-3651. TestQueueManagerRefresh fails. (Thomas Graves via mahadev)
  2667. MAPREDUCE-3645. TestJobHistory fails. (Thomas Graves via mahadev)
  2668. MAPREDUCE-3652. org.apache.hadoop.mapred.TestWebUIAuthorization.testWebUIAuthorization
  2669. fails. (Thomas Graves via mahadev)
  2670. MAPREDUCE-3625. CapacityScheduler web-ui display of queue's used capacity is broken.
  2671. (Jason Lowe via mahadev)
  2672. MAPREDUCE-3596. Fix scheduler to handle cleaned up containers, which NMs
  2673. may subsequently report as running. (Vinod Kumar Vavilapalli via sseth)
  2674. MAPREDUCE-3656. Fixed a race condition in MR AM which is failing the sort
  2675. benchmark consistently. (Siddarth Seth via vinodkv)
  2676. MAPREDUCE-3532. Modified NM to report correct http address when an ephemeral
  2677. web port is configured. (Bhallamudi Venkata Siva Kamesh via vinodkv)
  2678. MAPREDUCE-3404. Corrected MR AM to honor speculative configuration and enable
  2679. speculating either maps or reduces. (Eric Payne via vinodkv)
  2680. MAPREDUCE-3664. Federation Documentation has incorrect configuration example.
  2681. (Brandon Li via jitendra)
  2682. MAPREDUCE-3649. Job End notification gives an error on calling back.
  2683. (Ravi Prakash via mahadev)
  2684. MAPREDUCE-3657. State machine visualize build fails. (Jason Lowe
  2685. via mahadev)
  2686. MAPREDUCE-2450. Fixed a corner case with interrupted communication threads
  2687. leading to a long timeout in Task. (Rajesh Balamohan via acmurthy)
  2688. MAPREDUCE-3669. Allow clients to talk to MR HistoryServer using both
  2689. delegation tokens and kerberos. (mahadev via acmurthy)
  2690. MAPREDUCE-3684. LocalDistributedCacheManager does not shut down its thread
  2691. pool (tomwhite)
  2692. MAPREDUCE-3582. Move successfully passing MR1 tests to MR2 maven tree.
  2693. (ahmed via tucu)
  2694. MAPREDUCE-3698. Client cannot talk to the history server in secure mode.
  2695. (mahadev)
  2696. MAPREDUCE-3689. RM web UI doesn't handle newline in job name.
  2697. (Thomas Graves via mahadev)
  2698. MAPREDUCE-3701. Delete HadoopYarnRPC from 0.23 branch.
  2699. (mahadev)
  2700. MAPREDUCE-3549. write api documentation for web service apis for RM, NM,
  2701. mapreduce app master, and job history server (Thomas Graves via mahadev)
  2702. MAPREDUCE-3705. ant build fails on 0.23 branch. (Thomas Graves via
  2703. mahadev)
  2704. MAPREDUCE-3691. webservices add support to compress response.
  2705. (Thomas Graves via mahadev)
  2706. MAPREDUCE-3702. internal server error trying access application master
  2707. via proxy with filter enabled (Thomas Graves via mahadev)
  2708. MAPREDUCE-3646. Remove redundant URL info from "mapred job" output.
  2709. (Jonathan Eagles via mahadev)
  2710. MAPREDUCE-3681. Fixed computation of queue's usedCapacity. (acmurthy)
  2711. MAPREDUCE-3505. yarn APPLICATION_CLASSPATH needs to be overridable.
  2712. (ahmed via tucu)
  2713. MAPREDUCE-3714. Fixed EventFetcher and Fetcher threads to shut-down properly
  2714. so that reducers don't hang in corner cases. (vinodkv)
  2715. MAPREDUCE-3712. The mapreduce tar does not contain the hadoop-mapreduce-client-
  2716. jobclient-tests.jar. (mahadev)
  2717. MAPREDUCE-3717. JobClient test jar has missing files to run all the test programs.
  2718. (mahadev)
  2719. MAPREDUCE-3630. Fixes a NullPointer exception while running TeraGen - if a
  2720. map is asked to generate 0 records. (Mahadev Konar via sseth)
  2721. MAPREDUCE-3683. Fixed maxCapacity of queues to be product of parent
  2722. maxCapacities. (acmurthy)
  2723. MAPREDUCE-3713. Fixed the way head-room is allocated to applications by
  2724. CapacityScheduler so that it deducts current-usage per user and not
  2725. per-application. (Arun C Murthy via vinodkv)
  2726. MAPREDUCE-3721. Fixed a race in shuffle which caused reduces to hang.
  2727. (sseth via acmurthy)
  2728. MAPREDUCE-3733. Add Apache License Header to hadoop-distcp/pom.xml.
  2729. (mahadev)
  2730. MAPREDUCE-3735. Add distcp jar to the distribution (tar).
  2731. (mahadev)
  2732. MAPREDUCE-3720. Changed bin/mapred job -list to not print job-specific
  2733. information not available at RM. (vinodkv via acmurthy)
  2734. MAPREDUCE-3742. "yarn logs" command fails with ClassNotFoundException.
  2735. (Jason Lowe via mahadev)
  2736. MAPREDUCE-3703. ResourceManager should provide node lists in JMX output.
  2737. (Eric Payne via mahadev)
  2738. MAPREDUCE-3716. Fixing YARN+MR to allow MR jobs to be able to use
  2739. java.io.File.createTempFile to create temporary files as part of their
  2740. tasks. (Jonathan Eagles via vinodkv)
  2741. MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug.
  2742. (ramya via acmurthy)
  2743. MAPREDUCE-3764. Fixed resource usage metrics for queues and users.
  2744. (acmurthy)
  2745. MAPREDUCE-3749. ConcurrentModificationException in counter groups.
  2746. (tomwhite)
  2747. MAPREDUCE-3762. Fixed default CapacityScheduler configs. (mahadev via
  2748. acmurthy)
  2749. MAPREDUCE-3499. New MiniMR does not setup proxyuser configuration
  2750. correctly, thus tests using doAs do not work. (johnvijoe via tucu)
  2751. MAPREDUCE-3696. MR job via oozie does not work on hadoop 23.
  2752. (John George via mahadev)
  2753. MAPREDUCE-3427. Fix streaming unit tests broken after mavenization.
  2754. (Hitesh Shah via acmurthy)
  2755. MAPREDUCE-3640. Allow AMRecovery to work with partial JobHistory files.
  2756. (Arun C Murthy via sseth)
  2757. MAPREDUCE-3752. Modified application limits to include queue max-capacities
  2758. besides the usual user limits. (Arun C Murthy via vinodkv)
  2759. MAPREDUCE-3744. Fix the yarn logs command line. Improve error messages for
  2760. mapred job -logs. (Jason Lowe via sseth)
  2761. MAPREDUCE-3780. Fixed a bug where applications killed before getting
  2762. activated were not getting cleaned up properly. (Hitesh Shah via acmurthy)
  2763. MAPREDUCE-3708. Metrics: Incorrect Apps Submitted Count (Bhallamudi via
  2764. mahadev)
  2765. MAPREDUCE-3727. jobtoken location property in jobconf refers to wrong
  2766. jobtoken file (tucu)
  2767. MAPREDUCE-3711. Fixed MR AM recovery so that only single selected task
  2768. output is recovered and thus reduce the unnecessarily bloated recovery
  2769. time. (Robert Joseph Evans via vinodkv)
  2770. MAPREDUCE-3760. Changed active nodes list to not contain unhealthy nodes
  2771. on the webUI and metrics. (vinodkv)
  2772. MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and
  2773. JobHistoryServer web-apps. (Jonathan Eagles via vinodkv)
  2774. MAPREDUCE-3803. Fix broken build of raid contrib due to HDFS-2864.
  2775. (Ravi Prakash via suresh)
  2776. MAPREDUCE-3791. can't build site in hadoop-yarn-server-common.
  2777. (mahadev)
  2778. MAPREDUCE-3723. TestAMWebServicesJobs & TestHSWebServicesJobs
  2779. incorrectly asserting tests (Bhallamudi Venkata Siva Kamesh
  2780. via mahadev)
  2781. MAPREDUCE-3795. "job -status" command line output is malformed.
  2782. (vinodkv via mahadev)
  2783. MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when
  2784. there are a few unhealthy nodes (vinodkv via mahadev)
  2785. MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname.
  2786. (Hitesh Shah via mahadev)
  2787. MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-
  2788. allocation-mb setting (Hitesh Shah via mahadev)
  2789. MAPREDUCE-3747. Initialize queue metrics upfront and added start/finish
  2790. time to RM Web-UI. (acmurthy)
  2791. MAPREDUCE-3814. Fixed MRV1 compilation. (Arun C Murthy via vinodkv)
  2792. MAPREDUCE-3810. Performance tweaks - reduced logging in AM and defined
  2793. hascode/equals for ResourceRequest & Priority. (vinodkv via acmurthy)
  2794. MAPREDUCE-3813. Added a cache for resolved racks. (vinodkv via acmurthy)
  2795. MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps
  2796. but no reduces. (Robert Joseph Evans via vinodkv)
  2797. MAPREDUCE-3804. yarn webapp interface vulnerable to cross scripting attacks
  2798. (Dave Thompson via bobby)
  2799. MAPREDUCE-3354. Changed scripts so that jobhistory server is started by
  2800. bin/mapred instead of bin/yarn. (Jonathan Eagles via acmurthy)
  2801. MAPREDUCE-3809. Ensure that there is no needless sleep in Task at the end
  2802. of the task. (sseth via acmurthy)
  2803. MAPREDUCE-3794. Support mapred.Task.Counter and mapred.JobInProgress.Counter
  2804. enums for compatibility (Tom White via mahadev)
  2805. MAPREDUCE-3697. Support binary compatibility for Counters after
  2806. MAPREDUCE-901. (mahadev via acmurthy)
  2807. MAPREDUCE-3817. Fixed bin/mapred to allow running of distcp and archive
  2808. jobs. (Arpit Gupta via acmurthy)
  2809. MAPREDUCE-3709. TestDistributedShell is failing. (Hitesh Shah via
  2810. mahadev)
  2811. MAPREDUCE-3436. JobHistory webapp address should use the host configured
  2812. in the jobhistory address. (Ahmed Radwan via sseth)
  2813. MAPREDUCE-3815. Fixed MR AM to always use hostnames and never IPs when
  2814. requesting containers so that scheduler can give off data local containers
  2815. correctly. (Siddarth Seth via vinodkv)
  2816. MAPREDUCE-3833. Fixed a bug in reinitiaziling of queues. (Jason Lowe via
  2817. acmurthy)
  2818. MAPREDUCE-3826. Fixed a bug in RM web-ui which broke sorting. (Jonathan
  2819. Eagles via acmurthy)
  2820. MAPREDUCE-3823. Ensure counters are calculated only once after a job
  2821. finishes. (Vinod Kumar Vavilapalli via sseth)
  2822. MAPREDUCE-3827. Changed Counters to use ConcurrentSkipListMap for
  2823. performance. (vinodkv via acmurthy)
  2824. MAPREDUCE-3822. Changed FS counter computation to use all occurences of
  2825. the same FS scheme, instead of randomly using one. (Mahadev Konar via
  2826. sseth)
  2827. MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times
  2828. into the container request table when multiple hosts for a split happen to
  2829. be on the same rack. (Siddarth Seth via vinodkv)
  2830. MAPREDUCE-3828. Ensure that urls in single-node mode are correct. (sseth
  2831. via acmurthy)
  2832. MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)
  2833. MAPREDUCE-3843. Job summary log file found missing on the RM host
  2834. (Anupam Seth via tgraves)
  2835. MAPREDUCE-3846. Addressed MR AM hanging issues during AM restart and then
  2836. the recovery. (vinodkv)
  2837. MAPREDUCE-3802. Added test to validate that AM can crash multiple times and
  2838. still can recover successfully after MAPREDUCE-3846. (vinodkv)
  2839. MAPREDUCE-3858. Task attempt failure during commit results in task never completing.
  2840. (Tom White via mahadev)
  2841. MAPREDUCE-3856. Instances of RunningJob class givs incorrect job tracking
  2842. urls when mutiple jobs are submitted from same client jvm. (Eric Payne via
  2843. sseth)
  2844. MAPREDUCE-3880. Changed LCE binary to be 32-bit. (acmurthy)
  2845. Release 0.23.0 - 2011-11-01
  2846. INCOMPATIBLE CHANGES
  2847. MAPREDUCE-2455. Remove deprecated JobTracker.State in favour of
  2848. JobTrackerStatus. (tomwhite)
  2849. MAPREDUCE-2430. Remove mrunit contrib. (nigel via eli)
  2850. MAPREDUCE-2606. Remove IsolationRunner. (Alejandro Abdelnur via eli)
  2851. NEW FEATURES
  2852. MAPREDUCE-2682. Add "mapred classpath" command to print classpath
  2853. for MR applications. (vinodkv via acmurthy)
  2854. MAPREDUCE-2107. [Gridmix] Total heap usage emulation in Gridmix.
  2855. (Amar Kamat and Ravi Gummadi via amarrk)
  2856. MAPREDUCE-2106. [Gridmix] Cumulative CPU usage emulation in Gridmix.
  2857. (amarrk)
  2858. MAPREDUCE-2543. [Gridmix] High-Ram feature emulation in Gridmix. (amarrk)
  2859. MAPREDUCE-2408. [Gridmix] Compression emulation in Gridmix. (amarrk)
  2860. MAPREDUCE-2473. Add "mapred groups" command to query the server-side groups
  2861. resolved for a user. (Aaron T. Myers via todd)
  2862. MAPREDUCE-461. Enable ServicePlugins for the JobTracker.
  2863. (Fredrik Hedberg via tomwhite)
  2864. MAPREDUCE-2521. Create RPM and Debian packages for MapReduce. Changes
  2865. deployment layout to be consistent across the binary tgz, rpm, and deb.
  2866. (Eric Yang via omalley)
  2867. MAPREDUCE-2323. Add metrics to the fair scheduler. (todd)
  2868. MAPREDUCE-2037. Capture intermediate progress, CPU and memory usage for
  2869. tasks. (Dick King via acmurthy)
  2870. MAPREDUCE-279. MapReduce 2.0. Merging MR-279 branch into trunk. Contributed by
  2871. Arun C Murthy, Christopher Douglas, Devaraj Das, Greg Roelofs, Jeffrey
  2872. Naisbitt, Josh Wills, Jonathan Eagles, Krishna Ramachandran, Luke Lu, Mahadev
  2873. Konar, Robert Evans, Sharad Agarwal, Siddharth Seth, Thomas Graves, and Vinod
  2874. Kumar Vavilapalli.
  2875. MAPREDUCE-2930. Added the ability to be able to generate graphs from the
  2876. state-machine definitions. (Binglin Chang via vinodkv)
  2877. MAPREDUCE-2719. Add a simple, DistributedShell, application to illustrate
  2878. alternate frameworks on YARN. (Hitesh Shah via acmurthy)
  2879. MAPREDUCE-3104. Implemented Application-acls. (vinodkv)
  2880. MAPREDUCE-2708. Designed and implemented MR Application Master recovery to
  2881. make MR AMs resume their progress after restart. (Sharad Agarwal via vinodkv)
  2882. MAPREDUCE-2858. Added a WebApp Proxy for applications. (Robert Evans via
  2883. acmurthy)
  2884. IMPROVEMENTS
  2885. MAPREDUCE-2187. Reporter sends progress during sort/merge. (Anupam Seth via
  2886. acmurthy)
  2887. MAPREDUCE-2365. Add counters to track bytes (read,written) via
  2888. File(Input,Output)Format. (Siddharth Seth via acmurthy)
  2889. MAPREDUCE-2680. Display queue name in job client CLI. (acmurthy)
  2890. MAPREDUCE-2679. Minor changes to sync trunk with MR-279 branch. (acmurthy)
  2891. MAPREDUCE-2400. Remove Cluster's dependency on JobTracker via a
  2892. ServiceProvider for the actual implementation. (tomwhite via acmurthy)
  2893. MAPREDUCE-2596. [Gridmix] Summarize Gridmix runs. (amarrk)
  2894. MAPREDUCE-2563. [Gridmix] Add High-Ram emulation system tests to
  2895. Gridmix. (Vinay Kumar Thota via amarrk)
  2896. MAPREDUCE-2104. [Rumen] Add Cpu, Memory and Heap usages to
  2897. TraceBuilder's output. (amarrk)
  2898. MAPREDUCE-2554. [Gridmix] Add distributed cache emulation system tests
  2899. to Gridmix. (Vinay Kumar Thota via amarrk)
  2900. MAPREDUCE-2543. [Gridmix] High-Ram feature emulation testcase. (amarrk)
  2901. MAPREDUCE-2469. Task counters should also report the total heap usage of
  2902. the task. (Ravi Gummadi and Amar Ramesh Kamat via amarrk)
  2903. MAPREDUCE-2544. [Gridmix] Add compression emulation system tests to
  2904. Gridmix. (Vinay Kumar Thota via amarrk)
  2905. MAPREDUCE-2517. [Gridmix] Add system tests to Gridmix.
  2906. (Vinay Kumar Thota via amarrk)
  2907. MAPREDUCE-2492. The new MapReduce API should make available task's
  2908. progress to the task. (amarrk)
  2909. MAPREDUCE-2153. Bring in more job configuration properties in to the trace
  2910. file. (Rajesh Balamohan via amarrk)
  2911. MAPREDUCE-1461. Feature to instruct rumen-folder utility to skip jobs worth
  2912. of specific duration. (Rajesh Balamohan via amarrk)
  2913. MAPREDUCE-2172. Added test-patch.properties required by test-patch.sh
  2914. (nigel)
  2915. MAPREDUCE-2156. Raid-aware FSCK. (Patrick Kling via dhruba)
  2916. MAPREDUCE-2215. A more elegant FileSystem#listCorruptFileBlocks API
  2917. (RAID changes) (Patrick Kling via hairong)
  2918. MAPREDUCE-1831. BlockPlacement policy for HDFS-RAID.
  2919. (Scott Chen via dhruba)
  2920. MAPREDUCE-1906. Lower minimum heartbeat interval for TaskTracker
  2921. (Scott Carey and Todd Lipcon via todd)
  2922. MAPREDUCE-1382. MRAsyncDiscService should tolerate missing local.dir.
  2923. (Zheng Shao and tomwhite via tomwhite)
  2924. MAPREDUCE-2263. MapReduce side of HADOOP-6904: RPC compatibility.
  2925. (hairong)
  2926. MAPREDUCE-1706. Log RAID recoveries on HDFS. (schen)
  2927. MAPREDUCE-2334. Update BlockPlacementPolicyRaid for the new method
  2928. in BlockPlacementPolicy. (szetszwo)
  2929. MAPREDUCE-2254. Allow setting of end-of-record delimiter for
  2930. TextInputFormat (Ahmed Radwan via todd)
  2931. MAPREDUCE-1927. Unit test for HADOOP-6835 (concatenated gzip support).
  2932. (Greg Roelofs via tomwhite)
  2933. MAPREDUCE-2206. The task-cleanup tasks should be optional. (schen)
  2934. MAPREDUCE-2225. MultipleOutputs should not require the use of 'Writable'.
  2935. (Harsh J Chouraria via tomwhite)
  2936. MAPREDUCE-1811. Job.monitorAndPrintJob() should print status of the job
  2937. at completion. (Harsh J Chouraria via tomwhite)
  2938. MAPREDUCE-993. bin/hadoop job -events <jobid> <from-event-#> <#-of-events>
  2939. help message is confusing. (Harsh J Chouraria via tomwhite)
  2940. MAPREDUCE-2302. Add static factory methods in GaloisField. (schen)
  2941. MAPREDUCE-2351. mapred.job.tracker.history.completed.location should
  2942. support an arbitrary filesystem URI. (tomwhite)
  2943. MAPREDUCE-2239. BlockPlacementPolicyRaid should call getBlockLocations
  2944. only when necessary. (schen)
  2945. MAPREDUCE-2331. Add coverage of task graph servlet to fair scheduler system
  2946. test. (todd)
  2947. MAPREDUCE-2367. Allow using a file to exclude certain tests from build.
  2948. (todd)
  2949. MAPREDUCE-2202. Generalize CLITest structure and interfaces to faciliate
  2950. upstream adoption (e.g. for web or system testing). (cos)
  2951. MAPREDUCE-2420. JobTracker should be able to renew delegation token over
  2952. HTTP (Boris Shkolnik via jitendra)
  2953. MAPREDUCE-2474. Add docs to the new API Partitioner on how to access the
  2954. Job Configuration. (Harsh J Chouraria via todd)
  2955. MAPREDUCE-2475. Disable IPV6 for junit tests. (suresh srinivas via mahadev)
  2956. MAPREDUCE-2422. Removed unused internal methods from DistributedCache.
  2957. (tomwhite)
  2958. MAPREDUCE-2456. Log the reduce taskID and associated TaskTrackers with
  2959. failed fetch notifications in the JobTracker log.
  2960. (Jeffrey Naisbitt via cdouglas)
  2961. MAPREDUCE-869. Documentation for config to set map/reduce task environment
  2962. (Alejandro Abdelnur via todd)
  2963. MAPREDUCE-2410. Add entry to streaming FAQ about how streaming reducers
  2964. receive keys. (Harsh J Chouraria via todd)
  2965. MAPREDUCE-2499. MR part of HADOOP-7291. (eli)
  2966. MAPREDUCE-2497. Missing spaces in error messages. (eli)
  2967. MAPREDUCE-2502. JobSubmitter should use mapreduce.job.maps instead of
  2968. its deprecated equivalent. (eli via todd)
  2969. MAPREDUCE-2381. JobTracker instrumentation not consistent about error
  2970. handling. (Philip Zeyliger via tomwhite)
  2971. MAPREDUCE-2449. Allow for command line arguments when performing
  2972. "Run on Hadoop" action in Eclipse plugin. (Jeff Zemerick via todd)
  2973. MAPREDUCE-2483. Remove duplication of jars between Hadoop subprojects
  2974. from build artifacts. (Eric Yang via omalley)
  2975. MAPREDUCE-2372. TaskLogAppender mechanism shouldn't be set up in
  2976. log4j.properties (todd)
  2977. MAPREDUCE-2516. Rename webinterface.private.actions to
  2978. mapreduce.jobtracker.webinterface.trusted (Ari Rabkin via todd)
  2979. MAPREDUCE-2459. Cache HAR filesystem metadata. (Mac Yang via mahadev)
  2980. HADOOP-7259. Contrib modules should include the build.properties from
  2981. the enclosing hadoop directory. (omalley)
  2982. MAPREDUCE-2494. Order distributed cache deletions by LRU. (Robert Joseph
  2983. Evans via cdouglas)
  2984. MAPREDUCE-2452. Makes the cancellation of delegation tokens happen in a
  2985. separate thread. (ddas)
  2986. HADOOP-7106. Reorganize project SVN layout to "unsplit" the projects.
  2987. (todd, nigel)
  2988. MAPREDUCE-2249. Check the reflexive property of Counters objects when
  2989. comparing equality. (Devaraj K via todd)
  2990. MAPREDUCE-2623. Update ClusterMapReduceTestCase to use
  2991. MiniDFSCluster.Builder (Harsh J Chouraria via eli)
  2992. MAPREDUCE-2602. Allow setting of end-of-record delimiter for
  2993. TextInputFormat for the old API. (Ahmed Radwan via todd)
  2994. MAPREDUCE-2705. Permits parallel multiple task launches.
  2995. (Thomas Graves via ddas)
  2996. MAPREDUCE-2489. Jobsplits with random hostnames can make the queue
  2997. unusable (jeffrey naisbit via mahadev)
  2998. MAPREDUCE-2854. update INSTALL with config necessary run mapred on yarn.
  2999. (thomas graves via mahadev)
  3000. MAPREDUCE-2701. app/Job.java needs UGI for the user that launched it.
  3001. (Robert Evans via mahadev)
  3002. MAPREDUCE-2652. Enabled multiple NMs to be runnable on a single node by
  3003. making shuffle service port to be truely configurable. (Robert Evans via
  3004. vinodkv)
  3005. MAPREDUCE-2735. Add an applications summary log to ResourceManager.
  3006. (Thomas Graves via acmurthy)
  3007. MAPREDUCE-2697. Enhance CapacityScheduler to cap concurrently running
  3008. applications per-queue & per-user. (acmurthy)
  3009. Configuration changes:
  3010. add yarn.capacity-scheduler.maximum-am-resource-percent
  3011. MAPREDUCE-2774. Add startup message to ResourceManager & NodeManager on
  3012. startup. (Venu Gopala Rao via acmurthy)
  3013. MAPREDUCE-2655. Add audit logs to ResourceManager and NodeManager. (Thomas
  3014. Graves via acmurthy)
  3015. MAPREDUCE-2864. Normalize configuration variable names for YARN. (Robert
  3016. Evans via acmurthy)
  3017. MAPREDUCE-2690. Web-page for FifoScheduler. (Eric Payne via acmurthy)
  3018. MAPREDUCE-2711. Update TestBlockPlacementPolicyRaid for the new namesystem
  3019. and block management APIs. (szetszwo)
  3020. MAPREDUCE-2933. Change allocate call to return ContainerStatus for
  3021. completed containers rather than Container. (acmurthy)
  3022. MAPREDUCE-2675. Reformat JobHistory Server main page to be more
  3023. useful. (Robert Joseph Evans via vinodkv).
  3024. MAPREDUCE-2896. Simplify all apis to in
  3025. org.apache.hadoop.yarn.api.records.* to be get/set only. Added javadocs to
  3026. all public records. (acmurthy)
  3027. MAPREDUCE-2676. MR-279: JobHistory Job page needs reformatted. (Robert Evans via
  3028. mahadev)
  3029. MAPREDUCE-2899. Replace major parts of ApplicationSubmissionContext with a
  3030. ContainerLaunchContext (Arun Murthy via mahadev)
  3031. MAPREDUCE-2966. Added ShutDown hooks for MRV2 processes so that they can
  3032. gracefully exit. (Abhijit Suresh Shingate via vinodkv)
  3033. MAPREDUCE-2672. MR-279: JobHistory Server needs Analysis this job.
  3034. (Robert Evans via mahadev)
  3035. MAPREDUCE-2965. Streamlined the methods hashCode(), equals(), compareTo()
  3036. and toString() for all IDs. (Siddharth Seth via vinodkv)
  3037. MAPREDUCE-2726. Added job-file to the AM and JobHistoryServer web
  3038. interfaces. (Jeffrey Naisbitt via vinodkv)
  3039. MAPREDUCE-2880. Improve classpath-construction for mapreduce AM and
  3040. containers. (Arun C Murthy via vinodkv)
  3041. MAPREDUCE-3055. Simplified ApplicationAttemptId passing to
  3042. ApplicationMaster via environment variable. (vinodkv)
  3043. MAPREDUCE-3092. Removed a special comparator for JobIDs in JobHistory as
  3044. JobIDs are already comparable. (Devaraj K via vinodkv)
  3045. MAPREDUCE-3099. Add docs for setting up a single node MRv2 cluster.
  3046. (mahadev)
  3047. MAPREDUCE-3001. Added task-specific counters to AppMaster and JobHistory
  3048. web-UIs. (Robert Joseph Evans via vinodkv)
  3049. MAPREDUCE-3098. Fixed RM and MR AM to report YarnApplicationState and
  3050. application's FinalStatus separately. (Hitesh Shah via vinodkv)
  3051. MAPREDUCE-2889. Added documentation for writing new YARN applications.
  3052. (Hitesh Shah via acmurthy)
  3053. MAPREDUCE-3134. Added documentation the CapacityScheduler. (acmurthy)
  3054. MAPREDUCE-3013. Removed YarnConfiguration.YARN_SECURITY_INFO and its usage
  3055. as it doesn't affect security any more. (vinodkv)
  3056. MAPREDUCE-2907. Changed log level for various messages in ResourceManager
  3057. from INFO to DEBUG. (Ravi Prakash via vinodkv)
  3058. MAPREDUCE-2702. Added a new API in OutputCommitter for recovering
  3059. the outputs of tasks from a crashed job so as to support MR Application
  3060. Master recovery. (Sharad Agarwal and Arun C Murthy via vinodkv)
  3061. MAPREDUCE-2738. Added the missing cluster level statistics on the RM web
  3062. UI. (Robert Joseph Evans via vinodkv)
  3063. MAPREDUCE-2988. Reenabled TestLinuxContainerExecutor reflecting the
  3064. current NodeManager code. (Robert Joseph Evans via vinodkv)
  3065. MAPREDUCE-3161. Improved some javadocs and fixed some typos in
  3066. YARN. (Todd Lipcon via vinodkv)
  3067. MAPREDUCE-3148. Ported MAPREDUCE-2702 to old mapred api for aiding task
  3068. recovery. (acmurthy)
  3069. MAPREDUCE-3133. Running a set of methods in a Single Test Class.
  3070. (Jonathan Eagles via mahadev)
  3071. MAPREDUCE-3059. QueueMetrics do not have metrics for aggregate
  3072. containers-allocated and aggregate containers-released.
  3073. (Devaraj K via mahadev)
  3074. MAPREDUCE-3187. Add names for various unnamed threads in MR2.
  3075. (Todd Lipcon and Siddharth Seth via mahadev)
  3076. MAPREDUCE-3136. Added documentation for setting up Hadoop clusters in both
  3077. non-secure and secure mode for both HDFS & YARN. (acmurthy)
  3078. MAPREDUCE-3068. Added a whitelist of environment variables for containers
  3079. from the NodeManager and set MALLOC_ARENA_MAX for all daemons and
  3080. containers. (Chris Riccomini via acmurthy)
  3081. MAPREDUCE-3144. Augmented JobHistory with the information needed for
  3082. serving aggregated logs. (Siddharth Seth via vinodkv)
  3083. MAPREDUCE-3163. JobClient spews errors when killing MR2 job.
  3084. (mahadev)
  3085. MAPREDUCE-3239. Use new createSocketAddr API in MRv2 to give better
  3086. error messages on misconfig (Todd Lipcon via mahadev)
  3087. MAPREDUCE-2747. Cleaned up LinuxContainerExecutor binary sources and changed
  3088. the configuration to use yarn names. (Robert Joseph Evans via vinodkv)
  3089. MAPREDUCE-3205. Fix memory specifications to be physical rather than
  3090. virtual, allowing for a ratio between the two to be configurable. (todd
  3091. via acmurthy)
  3092. MAPREDUCE-2986. Fixed MiniYARNCluster to support multiple NodeManagers.
  3093. (Anupam Seth via vinodkv)
  3094. MAPREDUCE-2736. Remove unused contrib components dependent on MR1. (eli)
  3095. MAPREDUCE-2989. Modified JobHistory to link to task and AM logs from the
  3096. JobHistoryServer. (Siddharth Seth via vinodkv)
  3097. MAPREDUCE-3014. Rename and invert logic of '-cbuild' profile to 'native' and off
  3098. by default. (tucu)
  3099. MAPREDUCE-3171. normalize nodemanager native code compilation with common/hdfs
  3100. native. (tucu)
  3101. MAPREDUCE-3146. Added a MR specific command line to dump logs for a
  3102. given TaskAttemptID. (Siddharth Seth via vinodkv)
  3103. MAPREDUCE-3275. Added documentation for AM WebApp Proxy. (Robert Evans via
  3104. acmurthy)
  3105. MAPREDUCE-3322. Added a better index.html and an brief overview of YARN
  3106. architecture. (acmurthy)
  3107. OPTIMIZATIONS
  3108. MAPREDUCE-2026. Make JobTracker.getJobCounters() and
  3109. JobInProgress.getCounters() aquire locks in a shorter time period.
  3110. (Joydeep Sen Sarma via schen)
  3111. MAPREDUCE-2740. MultipleOutputs in new API creates needless
  3112. TaskAttemptContexts. (todd)
  3113. MAPREDUCE-901. Efficient framework counters. (llu via acmurthy)
  3114. MAPREDUCE-2629. Workaround a JVM class loading quirk which prevents
  3115. JIT compilation of inner classes methods in ReduceContextImpl.
  3116. BUG FIXES
  3117. MAPREDUCE-2603. Disable High-Ram emulation in system tests.
  3118. (Vinay Kumar Thota via amarrk)
  3119. MAPREDUCE-2539. Fixed NPE in getMapTaskReports in JobClient. (Robert Evans via
  3120. acmurthy)
  3121. MAPREDUCE-1978. Rumen TraceBuilder should provide recursive
  3122. input folder scanning.
  3123. MAPREDUCE-2416. Remove the restriction of specifying group names in
  3124. users-list file for Gridmix in RoundRobinUserResolver mode.
  3125. MAPREDUCE-2417. Fix Gridmix in RoundRobinUserResolver mode to
  3126. map testing/proxy users to unique users in a trace.
  3127. MAPREDUCE-2307. Exception thrown in Jobtracker logs, when the Scheduler
  3128. configured is FairScheduler. (Devaraj K via matei)
  3129. MAPREDUCE-2199. build is broken 0.22 branch creation. (cos)
  3130. MAPREDUCE-1752. Implement getFileBlockLocations in HarFilesystem.
  3131. (Patrick Kling via dhruba)
  3132. MAPREDUCE-2155. RaidNode should optionally use the mapreduce jobs to
  3133. fix missing blocks. (Patrick Kling via dhruba)
  3134. MAPREDUCE-1334. Fix TestIndexUpdater by ignoring _SUCCESS file in HDFS.
  3135. (Kay Kay via yhemanth)
  3136. MAPREDUCE-2232. Add missing methods to TestMapredGroupMappingServiceRefresh.
  3137. (Todd Lipcon via eli)
  3138. MAPREDUCE-2271. Fix TestSetupTaskScheduling failure on trunk.
  3139. (Liyin Liang via todd)
  3140. MAPREDUCE-2290. Fix compilation error in TestTaskCommit. (eli)
  3141. MAPREDUCE-2294. Fix compilation error in mumak. (todd)
  3142. MAPREDUCE-2300. Fix TestUmbilicalProtocolWithJobToken on trunk after
  3143. HADOOP-6904. (todd)
  3144. MAPREDUCE-2296. Fix references to misspelled method name
  3145. getProtocolSigature (todd)
  3146. MAPREDUCE-2311. Fix TestFairScheduler failure (schen)
  3147. MAPREDUCE-1996. API: Reducer.reduce() method detail misstatement.
  3148. (Harsh J Chouraria via tomwhite)
  3149. MAPREDUCE-2203. Wrong javadoc for TaskRunner's appendJobJarClasspaths
  3150. method. (Jingguo Yao via tomwhite)
  3151. MAPREDUCE-2074. Task should fail when symlink creation fails.
  3152. (Priyo Mustafi via tomwhite)
  3153. MAPREDUCE-1242. Chain APIs error misleading.
  3154. (Harsh J Chouraria via tomwhite)
  3155. MAPREDUCE-2379. Adds missing DistributedCache configurations in
  3156. mapred-default.xml (Todd Lipcon via amareshwari)
  3157. MAPREDUCE-2348. Disable mumak tests on trunk since they currently time out
  3158. (todd)
  3159. MAPREDUCE-2395. TestBlockFixer timing out on trunk. (Ramkumar Vadali via
  3160. todd)
  3161. MAPREDUCE-2426. Make TestFairSchedulerSystem fail with more verbose output
  3162. (todd)
  3163. MAPREDUCE-2448. NoSuchMethodError:
  3164. org.apache.hadoop.hdfs.TestDatanodeBlockScanner.corruptReplica(..) (eli)
  3165. MAPREDUCE-2460. Fix flaky test TestFairSchedulerSystem. (todd)
  3166. MAPREDUCE-2451. Log the details from health check script at the
  3167. JobTracker. (Thomas Graves via cdouglas)
  3168. MAPREDUCE-2467. HDFS-1052 changes break the raid contrib module in
  3169. MapReduce. (suresh srinivas via mahadev)
  3170. MAPREDUCE-2258. IFile reader closes stream and compressor in wrong order.
  3171. (todd via tomwhite)
  3172. MAPREDUCE-2518. The t flag is missing in distcp help message. (Wei Yongjun
  3173. via szetszwo)
  3174. MAPREDUCE-2514. Fix typo in TaskTracker ReinitTrackerAction log message.
  3175. (Jonathan Eagles via cdouglas)
  3176. MAPREDUCE-2490. Add logging to graylist and blacklist activity to aid
  3177. diagnosis of related issues. (Jonathan Eagles via cdouglas)
  3178. MAPREDUCE-2495. exit() the TaskTracker when the distributed cache cleanup
  3179. thread dies. (Robert Joseph Evans via cdouglas)
  3180. MAPREDUCE-2470. Fix NPE in RunningJobs::getCounters. (Robert Joseph Evans
  3181. via cdouglas)
  3182. MAPREDUCE-2536. Update FsShell -mv command usage in TestMRCLI. (Daryn
  3183. Sharp via szetszwo)
  3184. MAPREDUCE-2529. Add support for regex-based shuffle metric counting
  3185. exceptions. (Thomas Graves via cdouglas)
  3186. MAPREDUCE-2559. ant binary fails due to missing c++ lib dir. (eli)
  3187. MAPREDUCE-2573. Fix new findbugs warning introduced by MAPREDUCE-2494.
  3188. (Robert Joseph Evans via todd)
  3189. MAPREDUCE-2581. Spelling errors in log messages. (Tim Sell via eli)
  3190. MAPREDUCE-2588. Change raid to the new DataTransferProtocol API. (szetszwo)
  3191. MAPREDUCE-2576. Typo in comment in SimulatorLaunchTaskAction.java.
  3192. (Tim Sell via jghoman)
  3193. MAPREDUCE-2550. Fix bin/mapred to work properly from within a source
  3194. checkout (Eric Yang via todd)
  3195. MAPREDUCE-2620. Update RAID for HDFS-2087. (szetszwo)
  3196. MAPREDUCE-2624. Update RAID for HDFS-2107. (szetszwo)
  3197. MAPREDUCE-2670. Fixing spelling mistake in FairSchedulerServlet.java. (eli)
  3198. MAPREDUCE-2710. Update JobSubmitter.printTokens(..) for HDFS-2161.
  3199. (szetszwo)
  3200. MAPREDUCE-2409. DistributedCache maps files and archives to the same path,
  3201. despite semantic incompatibility. (Siddharth Seth via cdouglas)
  3202. MAPREDUCE-2575. TestMiniMRDFSCaching fails if test.build.dir is set
  3203. to something other than build/test (Thomas Graves via mahadev)
  3204. MAPREDUCE-2622. Remove the last remaining reference to the deprecated
  3205. configuration "io.sort.mb". (Harsh J Chouraria via todd)
  3206. MAPREDUCE-2732. Remove directly accessing FSNamesystem.LOG from
  3207. TestCopyFiles and TestDistCh. (szetszwo)
  3208. MAPREDUCE-2463. Job history files are not moved to done folder when job
  3209. history location is hdfs. (Devaraj K via szetszwo)
  3210. MAPREDUCE-2243. Close streams propely in a finally-block to avoid leakage
  3211. in CompletedJobStatusStore, TaskLog, EventWriter and TotalOrderPartitioner.
  3212. (Devaraj K via szetszwo)
  3213. MAPREDUCE-2741. Make ant build system work with hadoop-common JAR
  3214. generated by Maven. (Alejandro Abdelnur via tomwhite)
  3215. MAPREDUCE-2760. mapreduce.jobtracker.split.metainfo.maxsize typoed
  3216. in mapred-default.xml. (todd via eli)
  3217. MAPREDUCE-2797. Update mapreduce tests and RAID for HDFS-2239. (szetszwo)
  3218. MAPREDUCE-2805. Update RAID for HDFS-2241. (szetszwo)
  3219. MAPREDUCE-2837. Ported bug fixes from y-merge to prepare for MAPREDUCE-279
  3220. merge. (acmurthy)
  3221. MAPREDUCE-2541. Fixed a race condition in IndexCache.removeMap. (Binglin
  3222. Chang via acmurthy)
  3223. MAPREDUCE-2458. Rename sanitized pom.xml in build directory to work around IDE
  3224. bug (Luke Lu via mahadev)
  3225. MAPREDUCE-279. Fix in MR-279 branch. Clear application notification if sent once
  3226. to NodeManager (mahadev)
  3227. MAPREDUCE-2433. YARNApplicationConstants hard code app master jar version (Luke
  3228. Lu via mahadev)
  3229. MAPREDUCE-279. Fix in MR-279 branch. Implement restart for resource manager
  3230. phase 1 - Helper classes to store and restore the data structures. (mahadev)
  3231. MAPREDUCE-2414. Change MRv2 to use generic interfaces. (Siddharth Seth via
  3232. acmurthy)
  3233. MAPREDUCE-279. Fix in MR-279 branch. Implement health-checks for the node -
  3234. server side(ResourceManager) changes. (vinodkv)
  3235. MAPREDUCE-2405: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2)
  3236. (Greg Roelofs via mahadev)
  3237. MAPREDUCE-279. Fix in MR-279 branch. Implementing Containers' memory monitoring.
  3238. (vinodkv)
  3239. MAPREDUCE-2440. Name clashes in TypeConverter (luke via mahadev)
  3240. MAPREDUCE-279. Fix in MR-279 branch. Add fail count to the command line of the
  3241. application master. (mahadev)
  3242. MAPREDUCE-2424. Polish uber-AppMaster: add uber-AM counters and GUI indicators.
  3243. (Greg Roelofs via mahadev)
  3244. MAPREDUCE-2405. Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2).
  3245. (Greg Roelofs and Sharad Agarwal via mahadev)
  3246. MAPREDUCE-279. Fix in MR-279 branch. Fix YarnRemoteException to give more
  3247. details. (Siddharth Seth via mahadev)
  3248. MAPREDUCE-279. Fix in MR-279 branch. WebApp for Job History (Krishna
  3249. Ramachandran via mahadev)
  3250. MAPREDUCE-279. Fix in MR-279 branch. Install sanitized poms for downstream
  3251. sanity (Luke Lu via mahadev)
  3252. MAPREDUCE-279. Fix in MR-279 branch. Add HistoryCleanerService to Job History
  3253. server. (Krishna Ramachandran via sharad)
  3254. MAPREDUCE-279. Fix in MR-279 branch. Implement 'bin/mapred job -list' and
  3255. 'bin/mapred job -list-active-trackers'. (acmurthy)
  3256. MAPREDUCE-279. Fix in MR-279 branch. Implement 'bin/mapred queue [-info
  3257. [-showJobs]] [-list] and enhanced 'bin/mapred job -list' to show queue and
  3258. ApplicationMaster information. (acmurthy)
  3259. MAPREDUCE-279. Fix in MR-279 branch. Fixed computation of user-limits at
  3260. runtime. (acmurthy)
  3261. MAPREDUCE-279. Fix in MR-279 branch. Added functionality to refresh queues at
  3262. runtime via the 'bin/yarn rmadmin' command. (acmurthy)
  3263. MAPREDUCE-279. Fix in MR-279 branch. Added functionality to stop/start queues.
  3264. (acmurthy)
  3265. MAPREDUCE-279. Fix in MR-279 branch. Introducing web-UI for NodeManager and
  3266. linking it from RM UI. (vinodkv)
  3267. MAPREDUCE-279. Fix in MR-279 branch. Fix race condition in TestJobHistoryEvents
  3268. and TestJobHistoryParsing. (sharad)
  3269. MAPREDUCE-279. Fix in MR-279 branch. Add Containers' logs' view to NM UI and
  3270. link it from AM UI. (vinodkv)
  3271. MAPREDUCE-279. Fix in MR-279 branch. Add ACLs for queues and command-line
  3272. utilities for viewing them. (acmurthy)
  3273. MAPREDUCE-279. Fix in MR-279 branch. Recovery of MR Application Master from
  3274. failures. (sharad)
  3275. MAPREDUCE-279. Fix in MR-279 branch. Added support High-RAM applications in
  3276. CapacityScheduler. (acmurthy)
  3277. MAPREDUCE-279. Fix in MR-279 branch. Completing the ZooKeeper Store for
  3278. ResourceManager state. (mahadev)
  3279. MAPREDUCE-279. Fix in MR-279 branch. Reorient container localization to be
  3280. per-container rather than per-application. (cdouglas)
  3281. MAPREDUCE-279. Fix in MR-279 branch. Fix file creation in
  3282. JobHistoryEventHandler. (sharad)
  3283. MAPREDUCE-279. Fix in MR-279 branch. Disable ContainerMonitoring for non-linux
  3284. systems. (vinodkv)
  3285. MAPREDUCE-279. Fix in MR-279 branch. Fix container launch w/ inconsistent
  3286. credential file naming. (cdouglas)
  3287. MAPREDUCE-2434. Metrics for ResourceManager. (Luke Lu via acmurthy)
  3288. MAPREDUCE-279. Fix in MR-279 branch. RM Restart Phase 2 - Completed the recovery
  3289. of components in the RM (mahadev)
  3290. MAPREDUCE-279. Fix in MR-279 branch. Fix to send finish application event only
  3291. when the application is finished (mahadev)
  3292. MAPREDUCE-2462. Write job conf along with JobHistory, other minor improvements.
  3293. (Siddharth Seth via sharad)
  3294. MAPREDUCE-279. Fix in MR-279 branch. Implement 'delay scheduling' for better
  3295. locality in CapacityScheduler and improved high-ram applications. (acmurthy)
  3296. MAPREDUCE-279. Fix in MR-279 branch. Implement Job Acls in MR Application
  3297. Master. (sharad)
  3298. MAPREDUCE-279. Fix in MR-279 branch. Moving userlogs out of container work-dir
  3299. into a separate directory structure. (vinodkv)
  3300. MAPREDUCE-279. Fix in MR-279 branch. Completing RM Restart. Completed Phase 3 of
  3301. making sure events are logged and restored (mahadev)
  3302. MAPREDUCE-2468. Add metrics for NM Shuffle. (Luke Lu via cdouglas)
  3303. MAPREDUCE-279. Fix in MR-279 branch. Adding user log handling for YARN. Making
  3304. NM put the user-logs on DFS and providing log-dump tools. (vinodkv)
  3305. MAPREDUCE-279. Fix in MR-279 branch. Fixing three tight-loops in RM that are
  3306. causing high cpu-usage. (vinodkv)
  3307. MAPREDUCE-279. Fix in MR-279 branch. Replacing FileContext usage with FileSystem
  3308. to work around security authentication issues with FileContext against a secure
  3309. DFS. (vinodkv)
  3310. MAPREDUCE-279. Fix in MR-279 branch. Client reconnect to restarted AM. (sharad)
  3311. MAPREDUCE-279. Fix in MR-279 branch. Fix refreshProxy in ClientServiceDelegate.
  3312. (sharad)
  3313. MAPREDUCE-279. Fix in MR-279 branch. Fix Null Pointer in TestUberAM. (sharad)
  3314. MAPREDUCE-2478. Improve history server. (Siddharth Seth via sharad)
  3315. MAPREDUCE-279. Fix in MR-279 branch. Improved TestJobHistoryEvents and
  3316. TestJobHistoryParsing. (sharad)
  3317. MAPREDUCE-279. Fix in MR-279 branch. Fix NM to use multiple disks for local
  3318. files and the userlogs. (vinodkv)
  3319. MAPREDUCE-2480: MR App should not depend on hard coded version of shuffle (luke
  3320. lu via mahadev)
  3321. MAPREDUCE-279. Fix in MR-279 branch. Propagate error back to client in case of a
  3322. job submission failure (mahadev)
  3323. MAPREDUCE-279. Fix in MR-279 branch. Fix assembly to add mapreduce shell scripts
  3324. to the assembly package. (mahadev)
  3325. MAPREDUCE-279. Fix in MR-279 branch. Fix TestQueueMetrics. (Luke Lu via sharad)
  3326. MAPREDUCE-279. Fix in MR-279 branch. Removal of stale application-log dirs from
  3327. NM local disks. (vinodkv)
  3328. MAPREDUCE-279. Fix in MR-279 branch. Add license header and minor cleanup in
  3329. history server. (Siddharth Seth via sharad)
  3330. MAPREDUCE-279. Fix in MR-279 branch. Minor fix for install instructions.
  3331. (mahadev)
  3332. MAPREDUCE-279. Fix in MR-279 branch. Fix a race in MR task that was causing MR
  3333. containers to overwrite each other's job.xml. Also fix leaking attempt-dirs in
  3334. app-local-dir. (vinodkv)
  3335. MAPREDUCE-279. Fix in MR-279 branch. Adding valid state to ASM on a finish when
  3336. its already completed and also disble UberAM. (mahadev)
  3337. MAPREDUCE-279. Fix in MR-279 branch. Fixed CS user limits. (acmurthy)
  3338. MAPREDUCE-279. Fix in MR-279 branch. Fixed reservation's bad interaction with
  3339. delay scheduling in CS. (acmurthy)
  3340. MAPREDUCE-279. Fix in MR-279 branch. Support mapreduce old (0.20) APIs. (sharad)
  3341. MAPREDUCE-279. Fix in MR-279 branch. Support fail-fast for MR jobs. (ddas)
  3342. MAPREDUCE-279. Fix in MR-279 branch. Fix for clearing container requests on an
  3343. AM failure and add tostring methods to taskids and taskattemptids for better
  3344. grep support. (mahadev)
  3345. MAPREDUCE-279. Fix in MR-279 branch. Speed up communication between MR AM and RM
  3346. by relying on a new config rather than AM_EXPIRY_INTERVAL which is too large.
  3347. (acmurthy)
  3348. MAPREDUCE-279. Fix in MR-279 branch. Fix calculation of maximum capacity to use
  3349. parent's absolute-capacity rather than the leaf queue's absolute-capacity.
  3350. (acmurthy)
  3351. MAPREDUCE-279. Fix in MR-279 branch. Fixing a bug in JobIDPbImpl that's causing
  3352. AM to crash randomly. (vinodkv)
  3353. MAPREDUCE-279. Fix in MR-279 branch. Fix calculation of maximum capacity in
  3354. ParentQueue to use its parent's absolute-capacity rather than its own
  3355. absolute-capacity. (acmurthy)
  3356. MAPREDUCE-279. Fix in MR-279 branch. Launching bin/yarn and bin/mapred only
  3357. *once* in AM for constructing classpaths to avoid multiple forks and huge vmem
  3358. usage by AM. (vinodkv)
  3359. MAPREDUCE-279. Fix in MR-279 branch. Fix CapacityScheduler to release unused
  3360. reservations on application completion. (acmurthy)
  3361. MAPREDUCE-279. Fix in MR-279 branch. Fix CapacityScheduler (LeafQueue) to not
  3362. allocate DATA_LOCAL containers when they are not required on the rack.
  3363. (acmurthy)
  3364. MAPREDUCE-279. Fix in MR-279 branch. Makes uber-task disabled by default (ddas)
  3365. MAPREDUCE-279. Fix in MR-279 branch. Make logging and memory for AM configurable
  3366. for the user via command line (mahadev)
  3367. MAPREDUCE-279. Fix in MR-279 branch. Fixing a bug in previous patch (r1103657).
  3368. Now bin/yarn truly shouldn't be launched multiple times in a single AM.
  3369. (vinodkv)
  3370. MAPREDUCE-279. Fix in MR-279 branch. Fixing a bug to do with setting the staging
  3371. dir. (ddas)
  3372. MAPREDUCE-279. Fix in MR-279 branch. Fixed Composite Service to shutdown
  3373. services if an error occurs on starting any one of those (mahadev & chris)
  3374. MAPREDUCE-279. Fix in MR-279 branch. Fix the tests to use jvm fork mode to avoid
  3375. errors in shutting down services (sidharth seth)
  3376. MAPREDUCE-2500. PB factories are not thread safe (Siddharth Seth via mahadev)
  3377. MAPREDUCE-2504. race in JobHistoryEventHandler stop (Siddharth Seth via mahadev)
  3378. MAPREDUCE-279. Fix in MR-279 branch. Fix job hang if the AM launch fails.
  3379. (mahadev)
  3380. MAPREDUCE-2509. Fix NPE in UI for pending attempts. (Luke Lu via mahadev)
  3381. MAPREDUCE-279. Fix in MR-279 branch. Add junit jar to lib in assembly (mahadev
  3382. and luke)
  3383. MAPREDUCE-279. Fix in MR-279 branch. Distributed cache bug fix to pass Terasort.
  3384. (vinodkv)
  3385. MAPREDUCE-279. Fix in MR-279 branch. Fix null pointer exception in kill task
  3386. attempt (mahadev)
  3387. MAPREDUCE-279. Fix in MR-279 branch. Refactored RMContainerAllocator to release
  3388. unused containers. (sharad)
  3389. MAPREDUCE-279. Fix in MR-279 branch. Changed Scheduler to return available limit
  3390. to AM in the allocate api. (acmurthy)
  3391. MAPREDUCE-279. Fix in MR-279 branch. Fix nodemanager expiry to not throw OOM.
  3392. (mahadev)
  3393. MAPREDUCE-279. Fix in MR-279 branch. Use DefaultContainerExecutor for
  3394. integration tests. (cdouglas)
  3395. MAPREDUCE-279. Fix in MR-279 branch. Fix NPE in test case (mahadev)
  3396. MAPREDUCE-279. Fix in MR-279 branch. Fix for regression on the scheduling of
  3397. reduces before maps are done (ddas)
  3398. MAPREDUCE-279. Fix in MR-279 branch. Fix distributed-cache related bugs.
  3399. (vinodkv)
  3400. MAPREDUCE-279. Fix in MR-279 branch. Ensure unused containers released by AM are
  3401. correctly counted for queue-capacity. (acmurthy)
  3402. MAPREDUCE-279. Fix in MR-279 branch. Fix TestRuntimeEstimators (Siddharth Seth
  3403. via ddas)
  3404. MAPREDUCE-279. Fix in MR-279 branch. Fix queue refresh to correctly record newly
  3405. added queues in CapacityScheduler. (acmurthy)
  3406. MAPREDUCE-279. Fix in MR-279 branch. Added metrics for tracking reservations in
  3407. CapacityScheduler. (Luke Lu via acmurthy)
  3408. MAPREDUCE-2522. Security for JobHistory service. (Siddharth Seth via mahadev)
  3409. MAPREDUCE-2534. Fix CI breaking hard coded version in jobclient pom. (Luke Lu
  3410. via mahadev)
  3411. MAPREDUCE-279. Fix in MR-279 branch. Add public cache. (cdouglas)
  3412. MAPREDUCE-279. Fix in MR-279 branch. Made number of RPC server threads
  3413. configurable. (acmurthy)
  3414. MAPREDUCE-279. Fix in MR-279 branch. Added acl check for RMAdmin. (acmurthy)
  3415. MAPREDUCE-279. Fix in MR-279 branch. Adding job kill for any state that the job
  3416. is in with access control. (mahadev)
  3417. MAPREDUCE-279. Fix in MR-279 branch. Add debug statements for AM not launching
  3418. (mahadev)
  3419. MAPREDUCE-279. Fix in MR-279 branch. Fixing race condition leader to hung jobs
  3420. in scheduler negotiator (mahadev)
  3421. MAPREDUCE-279. Fix in MR-279 branch. Add debug config for delaying delete of
  3422. local files. (cdouglas)
  3423. MAPREDUCE-2527. Metrics for MRAppMaster (Luke lu via mahadev)
  3424. MAPREDUCE-2532. Metrics for NodeManager (Luke Lu via mahadev)
  3425. MAPREDUCE-279. Fix in MR-279 branch. Fixed an NPE during handling of unnecessary
  3426. reservations in CS. (acmurthy)
  3427. MAPREDUCE-279. Fix in MR-279 branch. Fix for public dist cache to work with non
  3428. default hdfs (mahadev &ddas)
  3429. MAPREDUCE-279. Fix in MR-279 branch. Making streaming -file option work. Also
  3430. minor fixes for successful compilation of contrib tests. (vinodkv)
  3431. MAPREDUCE-2536. Backporting changes to MR-279.
  3432. MAPREDUCE-279. Fix in MR-279 branch. Bugfix for using user staging directory for
  3433. history files (Siddharth Seth via mahadev)
  3434. MAPREDUCE-279. Fix in MR-279 branch. First fix for making basic speculative
  3435. execution work (ddas)
  3436. MAPREDUCE-279. Fix in MR-279 branch. Fixes for TestFail/Kill (ddas)
  3437. MAPREDUCE-279. Fix in MR-279 branch. Set correct version of avro-maven-plugin
  3438. that is available in apache maven repositories. (vinodkv)
  3439. MAPREDUCE-279. Fix in MR-279 branch. Fixing and reneabling
  3440. TestContainerTokenSecretManager. (vinodkv)
  3441. MAPREDUCE-279. Fix in MR-279 branch. Cleaning up configuration constants in
  3442. mapreduce modules. (vinodkv)
  3443. MAPREDUCE-279. Fix in MR-279 branch. Fixing NPE on speculator in MRAppMaster and
  3444. making job-history optional in tests to make test goal succeed. (vinodk and
  3445. sharadag).
  3446. MAPREDUCE-279. Fix in MR-279 branch. Fixed NPE in CS by checking Application
  3447. state before scheduling and fixing synchronization in CS. (acmurthy)
  3448. MAPREDUCE-279. Fix in MR-279 branch. Making pipes work with YARN. Changed pipes
  3449. to get log-locations from an environmental variable. (vinodkv)
  3450. MAPREDUCE-279. Fix in MR-279 branch. Ensure 'lost' NodeManagers are dealt
  3451. appropriately, the containers are released correctly. (acmurthy)
  3452. MAPREDUCE-279. Fix in MR-279 branch. Adding some more logging for AM expiry logs
  3453. (mahadev)
  3454. MAPREDUCE-279. Fix in MR-279 branch. Reduce ramp up and zero maps support.
  3455. (sharad)
  3456. MAPREDUCE-279. Fix in MR-279 branch. Allowing hdfs calls from streaming/pipes
  3457. tasks. (vinodkv)
  3458. MAPREDUCE-279. Fix in MR-279 branch. Added ability to decommission nodes and
  3459. completed RM administration tools to achieve parity with JobTracker. (acmurthy)
  3460. MAPREDUCE-2551. Added JobSummaryLog. (Siddharth Seth via acmurthy)
  3461. MAPREDUCE-2552. Fixed NPE in CompletedJob in JobHistoryServer. (Siddharth Seth
  3462. via acmurthy)
  3463. MAPREDUCE-279. Fix in MR-279 branch. Fix reduce slow start. (sharad)
  3464. MAPREDUCE-279. Fix in MR-279 branch. Fixed TestFifoScheduler. (acmurthy)
  3465. MAPREDUCE-279. Fix in MR-279 branch. Fix build issue for using yarn.version
  3466. instead of hadoop-mapred.version (mahadev and giri)
  3467. MAPREDUCE-279. Fix in MR-279 branch. Fixes in the handling of KILL events in the
  3468. SUCCEEDED state for tasks in the application master (ddas)
  3469. MAPREDUCE-279. Fix in MR-279 branch. Fix for NPE in TestRMNMRPCResponseId.
  3470. (mahadev)
  3471. MAPREDUCE-279. Fix in MR-279 branch. Fix a deadlock in the resourcemanager.
  3472. (mahadev)
  3473. MAPREDUCE-279. Fix in MR-279 branch. NodeStatus.getNodeHealthStatus().setBlah
  3474. broken (Siddharth Seth)
  3475. MAPREDUCE-279. Fix in MR-279 branch. Fix another NPE in TestRMNMRPCResponseId.
  3476. (mahadev)
  3477. MAPREDUCE-279. Fix in MR-279 branch. Fix for NPE in TestNMExpiry (siddharth
  3478. seth)
  3479. MAPREDUCE-279. Fix in MR-279 branch. Making each node aggregate all its
  3480. user-logs to a separate hdfs file. (vinodkv)
  3481. MAPREDUCE-279. Fix in MR-279 branch. Fix calculation of max-capacity for a
  3482. queue, also fixed a bug in registration of NodeManagers. (acmurthy)
  3483. MAPREDUCE-279. Fix in MR-279 branch. More cleaning up constants, removing stale
  3484. code, and making conspicuous the envs that apps depend on to be provided by
  3485. YARN. (vinodkv)
  3486. MAPREDUCE-279. Fix in MR-279 branch. Fix container size rounding in AM and
  3487. headroom in RM. (acmurthy and sharad)
  3488. MAPREDUCE-279. Fix in MR-279 branch. Disable Job acls until fixed (mahadev)
  3489. MAPREDUCE-279. Fix in MR-279 branch. Fix to report job status if the application
  3490. is KILLED/FAILED. (mahadev)
  3491. MAPREDUCE-279. Fix in MR-279 branch. Fix a corner case in headroom computation -
  3492. now reservations are taken into account and headroom is computed much later to
  3493. account for allocations/reservations. (acmurthy)
  3494. MAPREDUCE-2537. The RM writes its log to
  3495. yarn-mapred-resourcemanager-<RM_Host>.out (Robert Evans via mahadev)
  3496. MAPREDUCE-279. Fix in MR-279 branch. Fix logging for showing the state of job
  3497. (FAILED/KILLED/SUCCEEDED) when it completes (mahadev)
  3498. MAPREDUCE-279. Fix in MR-279 branch. Re-enabled TestCapacityScheduler.
  3499. (acmurthy)
  3500. MAPREDUCE-279. Fix in MR-279 branch. Support for min and max container capacity.
  3501. (acmurthy and sharad)
  3502. MAPREDUCE-2531. Fixed jobcontrol to downgrade JobID. (Robert Evans via acmurthy)
  3503. MAPREDUCE-2539. Fixed NPE in getMapTaskReports in JobClient. (Robert Evans via
  3504. acmurthy)
  3505. MAPREDUCE-279. Fix in MR-279 branch. Fixing the wrong config key used in
  3506. JobHistory that prevented configuring move-thread interval. (vinodkv)
  3507. MAPREDUCE-279. Fix in MR-279 branch. Fixed inconsistency in QueueACL enums.
  3508. (acmurthy)
  3509. MAPREDUCE-279. Fix in MR-279 branch. Fix various issues with Web UI's. (Luke Lu)
  3510. MAPREDUCE-279. Fix in MR-279 branch. Fix class cast exception in Task abort for
  3511. old mapreduce apis. (sharad)
  3512. MAPREDUCE-279. Fix in MR-279 branch. Add deletion of distributed cache
  3513. resources. (cdouglas)
  3514. MAPREDUCE-279. Fix in MR-279 branch. Disable aggregation of logs onto DFS till
  3515. JobHistoryServer starts serving logs. (vinodkv)
  3516. MAPREDUCE-279. Fix in MR-279 branch. Cleanup redundant code in TaskAttemptImpl.
  3517. (sharad)
  3518. MAPREDUCE-279. Fix in MR-279 branch. Work around broken signaling in public
  3519. cache. (cdouglas)
  3520. MAPREDUCE-2566. YarnConfiguration should reloadConfiguration if instantiated
  3521. with a non YarnConfiguration object. (Siddharth Seth)
  3522. MAPREDUCE-279. Fix in MR-279 branch. Fully resolve paths when launching
  3523. containers. (Siddharth Seth)
  3524. MAPREDUCE-279. Fix in MR-279 branch. Re-enabling Uber-AM feature. (vinodkv)
  3525. MAPREDUCE-279. Fix in MR-279 branch. Fixed deadlock during expiring NMs.
  3526. (acmurthy)
  3527. MAPREDUCE-279. Fix in MR-279 branch. Solving NPEs during
  3528. ContainerManager#StopContainer. Also removing the unused
  3529. ContainerManager#CleanupContainer api. (vinodkv)
  3530. MAPREDUCE-279. Fix in MR-279 branch. Remove retries in dist cache so that NM's
  3531. do not shutdown (mahadev)
  3532. MAPREDUCE-279. Fix in MR-279 branch. Fix classpath construction for Task.
  3533. (vinodkv via sharad)
  3534. MAPREDUCE-279. Fix in MR-279 branch. Sending Job diagnostics from AM to RM and
  3535. redirect to history-url on job completion. (vinodkv and sharadag)
  3536. MAPREDUCE-279. Fix in MR-279 branch. Added clover in pom dependency. (sharad)
  3537. MAPREDUCE-279. Fix in MR-279 branch. Intermittent TestMRApp failures on faster
  3538. Linux desktop. (Luke lu via mahadev)
  3539. MAPREDUCE-279. Fix in MR-279 branch. Including source files in release
  3540. distribution (Luke Lu via mahadev)
  3541. MAPREDUCE-279. Fix in MR-279 branch. Intermittent TestMRApp failures on faster
  3542. Linux desktop (part 2) (Luke lu via mahadev)
  3543. MAPREDUCE-279. Fix in MR-279 branch. Disable Uber AM. (mahadev)
  3544. MAPREDUCE-279. Fix in MR-279 branch. Added few job diagnostic messages. (sharad)
  3545. MAPREDUCE-279. Fix in MR-279 branch. Add ability to includes src files in
  3546. assembly target for maven (Luke Lu via mahadev)
  3547. MAPREDUCE-2582. Cleanup JobHistory event generation.(Siddharth Seth via sharad)
  3548. MAPREDUCE-279. Fix in MR-279 branch. Fix rounding off problem in reduce ramp up.
  3549. (sharad)
  3550. MAPREDUCE-279. Fix in MR-279 branch. Fix more rounding off problems in reduce
  3551. ramp up. Also fix a bug preventing the application of the cap on reduce ramp-up.
  3552. (Sharad Agarwal via vinodkv)
  3553. MAPREDUCE-279. Fix in MR-279 branch. Fix to exclude images dir into the tar
  3554. distribution (Luke Lu via gkesavan)
  3555. MAPREDUCE-279. Fix in MR-279 branch. Changes a couple of usages of FileContext
  3556. to FileSystem in YarnRunner to handle distributed cache path resolutions on
  3557. non-default filesystems. (ddas)
  3558. MAPREDUCE-279. Fix in MR-279 branch. Display failed/killed attempts of the task
  3559. on MR AM UI separately. (vinodkv)
  3560. MAPREDUCE-279. Fix in MR-279 branch. Miscellaneous UI fixes + source code
  3561. formatting for MR JobHistoryEventHandler. (vinodkv)
  3562. MAPREDUCE-279. Fix in MR-279 branch. Fixing broken link to logs for container on
  3563. NM web UI. (vinodkv)
  3564. MAPREDUCE-279. Fix in MR-279 branch. Fixing the bug which was causing FAILED
  3565. jobs to be displayed as COMPLETED on the RM UI. (vinodkv)
  3566. MAPREDUCE-279. Fix in MR-279 branch. Job level node blacklisting. (sharad)
  3567. MAPREDUCE-279. Fix in MR-279 branch. Fix NPE in history event handling
  3568. (Siddharth Seth via mahadev)
  3569. MAPREDUCE-2569. Ensure root queue allocated 100% capacity. (Jonathan Eagles via
  3570. cdouglas)
  3571. MAPREDUCE-279. Fix in MR-279 branch. Fix ClassCastException in JobHistoryServer
  3572. for certain jobs. (Siddharth Seth via llu)
  3573. MAPREDUCE-279. Fix in MR-279 branch. Changes for invoking rack resolution in the
  3574. RM and in the AM (ddas)
  3575. MAPREDUCE-279. Fix in MR-279 branch. Fix concurrent modification exception in
  3576. the Capacity Scheduler (mahadev)
  3577. MAPREDUCE-279. Fix in MR-279 branch. Fix stopContainer for setsid challenged
  3578. platforms. (llu)
  3579. MAPREDUCE-2587. Generate yarn version for UI. (Thomas Graves via lluts page to
  3580. the history server UI. (vinodkv)
  3581. MAPREDUCE-279. Fix in MR-279 branch. Bug fix to set correct state on containers
  3582. so as to avoid duplicate containers from RM to AM. (vinodkv)
  3583. MAPREDUCE-279. Fix in MR-279 branch. Hack until MAPREDUCE-2365 is fixed to make
  3584. PIG work with MRV2. (mahadev)
  3585. MAPREDUCE-279. Fix in MR-279 branch. Changes a couple of usages of FileContext
  3586. to FileSystem in TaskAttemptImpl to handle distributed cache path resolutions on
  3587. non-default filesystems. (ddas)
  3588. MAPREDUCE-279. Fix in MR-279 branch. Fix NPE when requesting attempts for
  3589. completed jobs. (Siddharth Seth via llu)
  3590. MAPREDUCE-279. Fix in MR-279 branch. Improve logging for AM when requesting
  3591. containers to show the right ask and release fields (mahadev)
  3592. MAPREDUCE-279. Fix in MR-279 branch. Fix race condition between multiple
  3593. localizers on a single node. (cdouglas via mahadev)
  3594. MAPREDUCE-279. Fix in MR-279 branch. Fix RM app start/finish time and
  3595. diagnostics. (llu)
  3596. MAPREDUCE-279. Fix in MR-279 branch. Fix to schedule reduces irrespective of the
  3597. headroom when all maps are done so as to avoid stall in reduce-scheduling when
  3598. slow-start is disabled. (Sharad Agarwal via vinodkv).
  3599. MAPREDUCE-279. Fix in MR-279 branch. Disabling locality-wait in
  3600. CapacityScheduler for now to prevent uber-slow scheduling for apps with no
  3601. data-locality constraints (sleep-job like). (vinodkv)
  3602. MAPREDUCE-279. Fix in MR-279 branch. Fixing scheduling deadlock in AM because of
  3603. incorrect headRoom values from RM. The bug happens when AM releases containers
  3604. and RM decrements current memory usage twice for all those containers. (vinodkv)
  3605. MAPREDUCE-2611. Fix counters, finish times etc. in job history. (Siddharth Seth
  3606. via llu)
  3607. MAPREDUCE-279. Fix in MR-279 branch. Fix for ConcurrentModification exception
  3608. while iterating through tokens in a UGI in ContainerLauncherImpl. (ddas)
  3609. MAPREDUCE-279. Fix in MR-279 branch. Fix for NPE in YarnChild that was causing
  3610. lots of tasks to fail. (vinodkv)
  3611. MAPREDUCE-2615. Make killJob go through AM and fix JobSummaryLog. (Siddharth
  3612. Seth via llu)
  3613. MAPREDUCE-279. Fix in MR-279 branch. Fix class cast exception in release
  3614. reserved containers in capacity scheduler (mahadev)
  3615. MAPREDUCE-279. Fix in MR-279 branch. Fix diagnostics display for more than 100
  3616. apps in RM. (llu)
  3617. MAPREDUCE-279. Fix in MR-279 branch. Fix some invalid transitions in the RM.
  3618. (vinodkv via ddas)
  3619. MAPREDUCE-2618. Fix NPE in 0 map 0 reduce jobs. (Jeffrey Naisbitt via llu)
  3620. MAPREDUCE-2625. Add version info to nodemanager info page. (Jonathan Eagles via
  3621. llu)
  3622. MAPREDUCE-279. Fix in MR-279 branch. (1) Faster retries from AM to HistoryServer
  3623. (2) Correct diagnostics for containers. (vinodkv)
  3624. MAPREDUCE-279. Fix in MR-279 branch. Major ASM cleanup. Streamlining classes,
  3625. interface and events. (vinodkv)
  3626. MAPREDUCE-279. Fix in MR-279 branch. Reinstate resolve path fixes for viewfs.
  3627. (Siddharth Seth via llu)
  3628. MAPREDUCE-2633. Add a getCounter(Enum) method to the Counters record. (Josh
  3629. Wills via sharad)
  3630. MAPREDUCE-2645. Updates to MRv2 INSTALL and README documentation. (Josh Wills
  3631. via vinodkv)
  3632. MAPREDUCE-2628. Add compiled on date to NM and RM info/about page.
  3633. MAPREDUCE-2400. Remove Cluster's dependency on JobTracker via a ServiceProvider
  3634. for the actual implementation. (tomwhite via acmurthy)
  3635. MAPREDUCE-2663. Refactoring StateMachineFactory inner classes. (ahmed radwan via
  3636. mahadev)
  3637. MAPREDUCE-2678. minimum-user-limit-percent no longer honored. (naisbitt via
  3638. mahadev)
  3639. MAPREDUCE-2630. refreshQueues leads to NPEs when used w/FifoScheduler. (Josh
  3640. Wills via mahadev)
  3641. MAPREDUCE-2644. NodeManager fails to create containers when NM_LOG_DIR is not
  3642. explicitly set in the Configuration. (Josh Wills via vinodkv)
  3643. MAPREDUCE-2661. Fix TaskImpl to not access MapTaskImpl. (Ahmed Radwan via
  3644. sharad)
  3645. HADOOP-6929. Backport changes to MR-279 (mahadev and owen)
  3646. HADOOP-6929. Making Security Info abstract and not an interface (mahadev)
  3647. MAPREDUCE-2667. mapred job -kill leaves application in RUNNING state (thomas
  3648. graves via mahadev)
  3649. MAPREDUCE-2664. Implement JobCounters for Mtions as asynchronous. (vinodkv,
  3650. sharad and acmurthy)
  3651. MAPREDUCE-2773. server.api.records.NodeHealthStatus renamed but not updated in
  3652. client NodeHealthStatus.java (Thomas Graves via mahadev)
  3653. MAPREDUCE-2772. Fix MR-279 build after common mavenization. (Robert Joseph Evans
  3654. via llu)
  3655. MAPREDUCE-2772. Fix MR-279 build after common mavenization, part 2. (Thomas
  3656. Graves via llu)
  3657. MAPREDUCE-279. Fix in MR-279 branch. Harmonize slf4j versions. (llu)
  3658. MAPREDUCE-279. Fix in MR-279 branch. Fix NPE in FifoScheduler. (mahadev)
  3659. MAPREDUCE-2776. Fix some of the yarn findbug warnings. (Siddharth Seth via
  3660. mahadev)
  3661. MAPREDUCE-279. Fix in MR-279 branch. Fix findbugs warnings in mr-client modules,
  3662. part 1 (mahadev)
  3663. MAPREDUCE-279. Fix in MR-279 branch. Fix findbugs warnings in mr-client modules
  3664. part 2 (mahadev)
  3665. MAPREDUCE-279. Fix in MR-279 branch. Fix findbugs warnings in mr-client modules
  3666. part 3 (mahadev)
  3667. MAPREDUCE-279. Fix in MR-279 branch. Fix the poms to enable 0.23 snapshots for
  3668. hdfs/common from apache nightly builds (gkesavan)
  3669. MAPREDUCE-279. Fix in MR-279 branch. Fix ivy conf to work with the hadoop common
  3670. trunk maven build changes. (Giridharan Kesavan)
  3671. MAPREDUCE-279. Fix in MR-279 branch. Patch for findbugs warnings in Resource
  3672. Manager (Siddharth Seth via mahadev)
  3673. MAPREDUCE-279. Fix in MR-279 branch. Fix for running ant targets to use the
  3674. right set of common/test jars (gkesavan via mahadev)
  3675. MAPREDUCE-2782. Unit tests for CapacityScheduler. (acmurthy)
  3676. MAPREDUCE-2706. Log job submission failures. (Jeffrey Naisbitt via acmurthy)
  3677. MAPREDUCE-2781. mr279 RM application finishtime not set (Thomas Graves via
  3678. mahadev)
  3679. MAPREDUCE-279. Fix in MR-279 branch. Fixed CS locality wait factor. (acmurthy)
  3680. MAPREDUCE-2808. pull MAPREDUCE-2797 into mr279 branch (Thomas Graves via
  3681. mahadev)
  3682. MAPREDUCE-2639. Bug fixes in speculate.DataStatistics. (Josh Wills via acmurthy)
  3683. MAPREDUCE-2839. Fixed TokenCache to get delegation tokens using both new and old
  3684. apis. (Siddharth Seth via acmurthy)
  3685. MAPREDUCE-2727. Fix divide-by-zero error in SleepJob for sleepCount equals 0.
  3686. (Jeffrey Naisbitt via acmurthy)
  3687. MAPREDUCE-2839. Fixed TokenCache to get delegation tokens using both new
  3688. and old apis. (Siddharth Seth via acmurthy)
  3689. MAPREDUCE-2727. Fix divide-by-zero error in SleepJob for sleepCount equals
  3690. 0. (Jeffrey Naisbitt via acmurthy)
  3691. MAPREDUCE-2860. Fix log4j logging in the maven test cases. (mahadev)
  3692. MAPREDUCE-2867. Remove Unused TestApplicaitonCleanup in resourcemanager/applicationsmanager.
  3693. (mahadev)
  3694. MAPREDUCE-2868. ant build broken in hadoop-mapreduce dir (mahadev, giri and arun via mahadev)
  3695. MAPREDUCE-2649. Handling of finished applications in RM. (Thomas Graves
  3696. via acmurthy)
  3697. MAPREDUCE-2838. Fix MapReduce builds to use new hadoop-common test jars.
  3698. (gkesavan via acmurthy)
  3699. MAPREDUCE-2859. Fix eclipse plugin contrib module compilation (gkesavan)
  3700. MAPREDUCE-2846. Fix missing synchronization in the task log management.
  3701. (omalley)
  3702. MAPREDUCE-2807. Fix AM restart and client redirection. (sharad)
  3703. MAPREDUCE-2877. Add missing Apache license header in some files in MR
  3704. and also add the rat plugin to the poms. (mahadev)
  3705. MAPREDUCE-2796. Set start times for MR applications for clients to see.
  3706. (Devaraj K via acmurthy)
  3707. MAPREDUCE-2879. Fix version for MR-279 to 0.23.0. (acmurthy)
  3708. MAPREDUCE-2881. Fix to include log4j 1.2.16 depenency (gkesavan)
  3709. MAPREDUCE-2885. Fix mapred-config.sh to look for hadoop-config.sh in
  3710. HADOOP_COMMON_HOME/libexec. (acmurthy)
  3711. MAPREDUCE-2893. Remove duplicate entry of YarnClientProtocolProvider in
  3712. ClientProtocolProvider services file. (Liang-Chi Hsieh via acmurthy)
  3713. MAPREDUCE-2891. Javadoc for AMRMProtocol and related records. (acmurthy)
  3714. MAPREDUCE-2898. Javadoc for ContainerManager protocol and related records.
  3715. (acmurthy)
  3716. MAPREDUCE-2904. Fixed bin/yarn to correctly include HDFS jars and
  3717. clean up of stale refs to pre-mavenized Hadoop Common and HDFS.
  3718. (Sharad Agarwal and Arun C. Murthy via acmurthy)
  3719. MAPREDUCE-2737. Update the progress of jobs on client side. (Siddharth Seth
  3720. and Mahadev Konar via mahadev)
  3721. MAPREDUCE-2886. Fix Javadoc warnings in MapReduce. (mahadev)
  3722. MAPREDUCE-2897. Javadoc for ClientRMProtocol protocol and related records.
  3723. (acmurthy)
  3724. MAPREDUCE-2916. Ivy build for MRv1 fails with bad organization for
  3725. common daemon. (mahadev)
  3726. MAPREDUCE-2917. Fixed corner case in container reservation which led to
  3727. starvation and hung jobs. (acmurthy)
  3728. MAPREDUCE-2756. Better error handling in JobControl for failed jobs.
  3729. (Robert Evans via acmurthy)
  3730. MAPREDUCE-2716. MRReliabilityTest job fails because of missing
  3731. job-file. (Jeffrey Naisbitt via vinodkv)
  3732. MAPREDUCE-2882. TestLineRecordReader depends on ant jars. (todd)
  3733. MAPREDUCE-2687. Fix NodeManager to use the right version of
  3734. LocalDirAllocator.getLocalPathToWrite. (mahadev & acmurthy)
  3735. MAPREDUCE-2800. Set final progress for tasks to ensure all task information
  3736. is correctly logged to JobHistory. (Siddharth Seth via acmurthy)
  3737. MAPREDUCE-2938. Log application submission failure in CapacityScheduler.
  3738. (acmurthy)
  3739. MAPREDUCE-2948. Hadoop streaming test failure, post MR-2767 (mahadev)
  3740. MAPREDUCE-2908. Fix all findbugs warnings. (vinodkv via acmurthy)
  3741. MAPREDUCE-2947. Fixed race condition in AuxiliaryServices. (vinodkv via
  3742. acmurthy)
  3743. MAPREDUCE-2844. Fixed display of nodes in UI. (Ravi Teja Ch N V via
  3744. acmurthy)
  3745. MAPREDUCE-2677. Fixed 404 for some links from HistoryServer. (Robert Evans
  3746. via acmurthy)
  3747. MAPREDUCE-2937. Ensure reason for application failure is displayed to the
  3748. user. (mahadev via acmurthy)
  3749. MAPREDUCE-2953. Fix a race condition on submission which caused client to
  3750. incorrectly assume application was gone by making submission synchronous
  3751. for RMAppManager. (Thomas Graves via acmurthy)
  3752. MAPREDUCE-2963. Fix hang in TestMRJobs. (Siddharth Seth via acmurthy)
  3753. MAPREDUCE-2954. Fixed a deadlock in NM caused due to wrong synchronization
  3754. in protocol buffer records. (Siddharth Seth via vinodkv)
  3755. MAPREDUCE-2975. Fixed YARNRunner to use YarnConfiguration rather than
  3756. Configuration. (mahadev via acmurthy)
  3757. MAPREDUCE-2971. ant build mapreduce fails protected access jc.displayJobList
  3758. (jobs) (Thomas Graves via mahadev)
  3759. MAPREDUCE-2691. Finishing up the cleanup of distributed cache file resources
  3760. and related tests. (Siddharth Seth via vinodkv)
  3761. MAPREDUCE-2749. Ensure NM registers with RM after starting all its services
  3762. correctly. (Thomas Graves via acmurthy)
  3763. MAPREDUCE-2979. Removed the needless ClientProtocolProvider configuration
  3764. from the hadoop-mapreduce-client-core module. (Siddharth Seth via vinodkv)
  3765. MAPREDUCE-2985. Fixed findbugs warnings in ResourceLocalizationService.
  3766. (Thomas Graves via acmurthy)
  3767. MAPREDUCE-2874. Fix formatting of ApplicationId in web-ui. (Eric Payne via
  3768. acmurthy)
  3769. MAPREDUCE-2995. Better handling of expired containers in MapReduce
  3770. ApplicationMaster. (vinodkv via acmurthy)
  3771. MAPREDUCE-2995. Fixed race condition in ContainerLauncher. (vinodkv via
  3772. acmurthy)
  3773. MAPREDUCE-2949. Fixed NodeManager to shut-down correctly if a service
  3774. startup fails. (Ravi Teja via vinodkv)
  3775. MAPREDUCE-3005. Fix both FifoScheduler and CapacityScheduler to correctly
  3776. enforce locality constraints. (acmurthy)
  3777. MAPREDUCE-3007. Fixed Yarn Mapreduce client to be able to connect to
  3778. JobHistoryServer in secure mode. (vinodkv)
  3779. MAPREDUCE-2987. Fixed display of logged user on RM Web-UI. (Thomas Graves
  3780. via acmurthy)
  3781. MAPREDUCE-3006. Fixed MapReduce AM to exit only after properly writing out
  3782. history file. (vinodkv)
  3783. MAPREDUCE-2925. Fixed Yarn+MR client code to behave saner with completed
  3784. jobs. (Devaraj K via vinodkv)
  3785. MAPREDUCE-3030. Fixed a bug in NodeId.equals() that was causing RM to
  3786. reject all NMs. (Devaraj K via vinodkv)
  3787. MAPREDUCE-3042. Fixed default ResourceTracker address. (Chris Riccomini
  3788. via acmurthy)
  3789. MAPREDUCE-3038. job history server not starting because conf() missing
  3790. HsController (Jeffrey Naisbitt via mahadev)
  3791. MAPREDUCE-3004. Fix ReduceTask to not assume 'local' mode in YARN. (Hitesh
  3792. Shah via acmurthy)
  3793. MAPREDUCE-3017. The Web UI shows FINISHED for killed/successful/failed jobs.
  3794. (mahadev)
  3795. MAPREDUCE-3040. Fixed extra copy of Configuration in
  3796. YarnClientProtocolProvider and ensured MiniMRYarnCluster sets JobHistory
  3797. configuration for tests. (acmurthy)
  3798. MAPREDUCE-3018. Fixed -file option for streaming. (mahadev via acmurthy)
  3799. MAPREDUCE-3036. Fixed metrics for reserved resources in CS. (Robert Evans
  3800. via acmurthy)
  3801. MAPREDUCE-2998. Fixed a bug in TaskAttemptImpl which caused it to fork
  3802. bin/mapred too many times. (vinodkv via acmurthy)
  3803. MAPREDUCE-3023. Fixed clients to display queue state correctly. (Ravi
  3804. Prakash via acmurthy)
  3805. MAPREDUCE-2970. Fixed NPEs in corner cases with different configurations
  3806. for mapreduce.framework.name. (Venu Gopala Rao via vinodkv)
  3807. MAPREDUCE-3062. Fixed default RMAdmin address. (Chris Riccomini
  3808. via acmurthy)
  3809. MAPREDUCE-3066. Fixed default ResourceTracker address for the NodeManager.
  3810. (Chris Riccomini via acmurthy)
  3811. MAPREDUCE-3044. Pipes jobs stuck without making progress. (mahadev)
  3812. MAPREDUCE-2754. Fixed MR AM stdout, stderr and syslog to redirect to
  3813. correct log-files. (Ravi Teja Ch N V via vinodkv)
  3814. MAPREDUCE-3073. Fixed build issues in MR1. (mahadev via acmurthy)
  3815. MAPREDUCE-2691. Increase threadpool size for launching containers in
  3816. MapReduce ApplicationMaster. (vinodkv via acmurthy)
  3817. MAPREDUCE-2990. Fixed display of NodeHealthStatus. (Subroto Sanyal via
  3818. acmurthy)
  3819. MAPREDUCE-3053. Better diagnostic message for unknown methods in ProtoBuf
  3820. RPCs. (vinodkv via acmurthy)
  3821. MAPREDUCE-2952. Fixed ResourceManager/MR-client to consume diagnostics
  3822. for AM failures in a couple of corner cases. (Arun C Murthy via vinodkv)
  3823. MAPREDUCE-3064. 27 unit test failures with Invalid
  3824. "mapreduce.jobtracker.address" configuration value for
  3825. JobTracker: "local" (Venu Gopala Rao via mahadev)
  3826. MAPREDUCE-3090. Fix MR AM to use ApplicationAttemptId rather than
  3827. (ApplicationId, startCount) consistently. (acmurthy)
  3828. MAPREDUCE-2646. Fixed AMRMProtocol to return containers based on
  3829. priority. (Sharad Agarwal and Arun C Murthy via vinodkv)
  3830. MAPREDUCE-3031. Proper handling of killed containers to prevent stuck
  3831. containers/AMs on an external kill signal. (Siddharth Seth via vinodkv)
  3832. MAPREDUCE-2984. Better error message for displaying completed containers.
  3833. (Devaraj K via acmurthy)
  3834. MAPREDUCE-3071. app master configuration web UI link under the Job menu
  3835. opens up application menu. (thomas graves via mahadev)
  3836. MAPREDUCE-3067. Ensure exit-code is set correctly for containers. (Hitesh
  3837. Shah via acmurthy)
  3838. MAPREDUCE-2999. Fix YARN webapp framework to properly filter servlet
  3839. paths. (Thomas Graves via vinodkv)
  3840. MAPREDUCE-3095. fairscheduler ivy including wrong version for hdfs.
  3841. (John George via mahadev)
  3842. MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev)
  3843. MAPREDUCE-3021. Change base urls for RM web-ui. (Thomas Graves via
  3844. acmurthy)
  3845. MAPREDUCE-3041. Fixed ClientRMProtocol to provide min/max resource
  3846. capabilities along-with new ApplicationId for application submission.
  3847. (Hitesh Shah via acmurthy)
  3848. MAPREDUCE-2843. Fixed the node-table to be completely displayed and making
  3849. node entries on RM UI to be sortable. (Abhijit Suresh Shingate via vinodkv)
  3850. MAPREDUCE-3110. Fixed TestRPC failure. (vinodkv)
  3851. MAPREDUCE-3078. Ensure MapReduce AM reports progress correctly for
  3852. displaying on the RM Web-UI. (vinodkv via acmurthy)
  3853. MAPREDUCE-3114. Fixed invalid ApplicationURL on RM WebUI. (Subroto Sanyal
  3854. via vinodkv)
  3855. MAPREDUCE-2791. Added missing info on 'job -status' output. (Devaraj K via
  3856. acmurthy)
  3857. MAPREDUCE-2996. Add uber-ness information to JobHistory. (Jonathan Eagles
  3858. via acmurthy)
  3859. MAPREDUCE-3050. Add ability to get resource usage information for
  3860. applications and nodes. (Robert Evans via acmurthy)
  3861. MAPREDUCE-3113. Ensure bin/yarn and bin/yarn-daemon.sh identify the root
  3862. of the install properly. (Xie Xianshan via acmurthy)
  3863. MAPREDUCE-3137. Fix broken merge of MAPREDUCE-2179. (Hitesh Shah via
  3864. acmurthy)
  3865. MAPREDUCE-2792. Replace usage of node ip-addresses with hostnames.
  3866. (vinodkv via acmurthy)
  3867. MAPREDUCE-3112. Fixed recursive sourcing of HADOOP_OPTS environment
  3868. variable. (Eric Yang)
  3869. MAPREDUCE-3056. Changed the default staging directory to not include
  3870. user.name to prevent issues with non-secure mode. (Devaraj K via vinodkv)
  3871. MAPREDUCE-2913. Fixed TestMRJobs.testFailingMapper to assert the correct
  3872. TaskCompletionEventStatus. (Jonathan Eagles via vinodkv)
  3873. MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per
  3874. queue per user. (John George via mahadev)
  3875. MAPREDUCE-2783. Fixing RM web-UI to show no tracking-URL when AM
  3876. crashes. (Eric Payne via vinodkv)
  3877. MAPREDUCE-3141. Fix the broken MRAppMaster to work over YARN in security
  3878. mode.(vinodkv)
  3879. MAPREDUCE-2751. Modified NodeManager to stop leaving around local files
  3880. after application finishes. (Siddharth Seth via vinodkv)
  3881. MAPREDUCE-3033. Ensure Master interface pays attention to classic v/s yarn
  3882. frameworks. (Hitesh Shah via acmurthy)
  3883. MAPREDUCE-2802. Ensure JobHistory filenames have jobId. (Jonathan Eagles
  3884. via acmurthy)
  3885. MAPREDUCE-2876. Use a different config for ContainerAllocationExpirer.
  3886. (Anupam Seth via acmurthy)
  3887. MAPREDUCE-3153. Fix TestFileOutputCommitter which was broken by
  3888. MAPREDUCE-2702. (mahadev via acmurthy)
  3889. MAPREDUCE-3123. Fix NM to quote symlink names to escape special
  3890. characters. (Hitesh Shah via acmurthy)
  3891. MAPREDUCE-3154. Fix JobSubmitter to check for output specs before copying
  3892. job submission files to fail fast. (Abhijit Suresh Shingate via acmurthy)
  3893. MAPREDUCE-3158. Fix test failures in MRv1 due to default framework being
  3894. set to yarn. (Hitesh Shah via acmurthy)
  3895. MAPREDUCE-3167. container-executor is not being packaged with the assembly
  3896. target. (mahadev)
  3897. MAPREDUCE-3020. Fixed TaskAttemptImpl to log the correct node-address for
  3898. a finished Reduce task. (Chackaravarthy via vinodkv)
  3899. MAPREDUCE-2668. Fixed AuxServices to send a signal on application-finish
  3900. to all the services. (Thomas Graves via vinodkv)
  3901. MAPREDUCE-3126. Fixed a corner case in CapacityScheduler where headroom
  3902. wasn't updated on changes to cluster size. (acmurthy)
  3903. MAPREDUCE-3140. Fixed the invalid JobHistory URL for failed
  3904. applications. (Subroto Sanyal via vinodkv)
  3905. MAPREDUCE-3125. Modified TaskImpl to consider only non-failed, non-killed
  3906. task-attempts for obtaining task's progress. (Hitesh Shah via vinodkv)
  3907. MAPREDUCE-2666. Retrieve shuffle port number from JobHistory on MR AM
  3908. restart. (Jonathan Eagles via acmurthy)
  3909. MAPREDUCE-2789. Complete schedulingInfo on CLI. (Eric Payne via acmurthy)
  3910. MAPREDUCE-3170. Fixed job output commit for deep hierarchies. (Hitesh Shah
  3911. via acmurthy)
  3912. MAPREDUCE-3124. Fixed location of native libs i.e. libhadoop.so for
  3913. containers. (John George via acmurthy)
  3914. MAPREDUCE-3057. Job History Server goes of OutOfMemory with 1200 Jobs
  3915. and Heap Size set to 10 GB. (Eric Payne via mahadev)
  3916. MAPREDUCE-2840. mr279 TestUberAM.testSleepJob test fails. (jonathan eagles
  3917. via mahadev)
  3918. MAPREDUCE-3190. Ensure bin/yarn fails early with a clear error message
  3919. when HADOOP_COMMON_HOME or HADOOP_HDFS_HOME are not set. (todd & acmurthy
  3920. via acmurthy)
  3921. MAPREDUCE-3189. Add link decoration back to MR2's CSS. (Todd Lipcon via
  3922. mahadev)
  3923. MAPREDUCE-3127. Changed default value of yarn.resourcemanager.acl.enable
  3924. to true and added some more documentation. (acmurthy)
  3925. MAPREDUCE-3032. Fixed TaskAttemptImpl so that JobHistory can have error
  3926. information about failed tasks. (Devaraj K via vinodkv)
  3927. MAPREDUCE-3196. TestLinuxContainerExecutorWithMocks fails on Mac OSX.
  3928. (Arun Murthy via mahadev)
  3929. MAPREDUCE-3197. TestMRClientService failing on building clean checkout of
  3930. branch 0.23 (mahadev)
  3931. MAPREDUCE-2762. Cleanup MR staging directory on completion. (mahadev via
  3932. acmurthy)
  3933. MAPREDUCE-3165. Ensure logging options are set correctly for MR AM and
  3934. tasks. (todd via acmurthy)
  3935. MAPREDUCE-3203. Fix some javac warnings in MRAppMaster. (mahadev)
  3936. MAPREDUCE-3162. Separated application-init and container-init event types
  3937. in NodeManager's Application state machine. (Todd Lipcon via vinodkv)
  3938. MAPREDUCE-3176. Fixed ant mapreduce tests that are timing out because
  3939. of wrong framework name. (Hitesh Shah via vinodkv)
  3940. MAPREDUCE-3181. Fixed MapReduce runtime to load yarn-default.xml and
  3941. yarn-site.xml. (acmurthy)
  3942. MAPREDUCE-2788. Normalize resource requests in FifoScheduler
  3943. appropriately. (Ahmed Radwan via acmurthy)
  3944. MAPREDUCE-2693. Fix NPE in job-blacklisting. (Hitesh Shah via acmurthy)
  3945. MAPREDUCE-3208. Fix NPE task/container log appenders. (liangzhwa via
  3946. acmurthy)
  3947. MAPREDUCE-3212. Fix usage/help message for bin/yarn. (Bhallamudi Venkata
  3948. Siva Kamesh via acmurthy)
  3949. MAPREDUCE-3179. Ensure failed tests exit with right error code. (Jonathan
  3950. Eagles via acmurthy)
  3951. MAPREDUCE-3188. Ensure correct shutdown in services. (todd via acmurthy)
  3952. MAPREDUCE-3226. Fix shutdown of fetcher threads. (vinodkv via acmurthy)
  3953. MAPREDUCE-3070. Fix NodeManager to use ephemeral ports by default.
  3954. (Devaraj K via acmurthy)
  3955. MAPREDUCE-3242. Trunk compilation broken with bad interaction from
  3956. MAPREDUCE-3070 and MAPREDUCE-3239. (mahadev)
  3957. MAPREDUCE-3058. Fixed MR YarnChild to report failure when task throws an
  3958. error and thus prevent a hanging task and job. (vinodkv)
  3959. MAPREDUCE-3087. Fixed the mapreduce classpath to correctly include the
  3960. generated-classpath file needed for tests. (Ravi Prakash via vinodkv)
  3961. MAPREDUCE-3233. Fixed a bug in MR Job so as to be able to restart the
  3962. application on AM crash. (Mahadev Konar via vinodkv)
  3963. MAPREDUCE-3028. Added job-end notification support. (Ravi Prakash via
  3964. acmurthy)
  3965. MAPREDUCE-3249. Ensure shuffle-port is correctly used duringMR AM recovery.
  3966. (vinodkv via acmurthy)
  3967. MAPREDUCE-3252. Fix map tasks to not rewrite data an extra time when
  3968. map output fits in spill buffer. (todd)
  3969. MAPREDUCE-3159. Ensure DefaultContainerExecutor doesn't delete application
  3970. directories during app-init. (todd via acmurthy)
  3971. MAPREDUCE-2746. Yarn servers can't communicate with each other with
  3972. hadoop.security.authorization set to true (acmurthy via mahadev)
  3973. MAPREDUCE-2821. Added missing fields (resourcePerMap & resourcePerReduce)
  3974. to JobSummary logs. (mahadev via acmurthy)
  3975. MAPREDUCE-3253. Fixed ContextFactory to clone JobContext correctly.
  3976. (acmurthy)
  3977. MAPREDUCE-3263. Fixed the MAPREDUCE-3028 commit which broke MR1. (Hitesh
  3978. Shah via acmurthy)
  3979. MAPREDUCE-3269. Fixed log4j properties to correctly set logging options
  3980. for JobHistoryServer vis-a-vis JobSummary logs. (mahadev via acmurthy)
  3981. MAPREDUCE-3250. When AM restarts, client keeps reconnecting to the new AM
  3982. and prints a lots of logs. (vinodkv via mahadev)
  3983. MAPREDUCE-3254. Fixed streaming to set the job.jar by using the right
  3984. JobConf ctor. (acmurthy)
  3985. MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically.
  3986. (acmurthy via mahadev)
  3987. MAPREDUCE-3175. Add authorization to admin web-pages such as /stacks, /jmx
  3988. etc. (Jonathan Eagles via acmurthy)
  3989. MAPREDUCE-3257. Added authorization checks for the protocol between
  3990. ResourceManager and ApplicationMaster. (vinodkv via acmurthy)
  3991. MAPREDUCE-3259. Added java.library.path of NodeManager to
  3992. ContainerLocalizer in LinuxContainerExecutor. (Kihwal Lee via acmurthy)
  3993. MAPREDUCE-3279. Fixed TestJobHistoryParsing which assumed user name to be
  3994. mapred all the time. (Siddharth Seth via acmurthy)
  3995. MAPREDUCE-3240. Fixed NodeManager to be able to forcefully cleanup its
  3996. containers (process-trees) irrespective of whether the container succeeded,
  3997. or killed. Contributed by Hitesh Shah.
  3998. MAPREDUCE-3281. Fixed a bug in TestLinuxContainerExecutorWithMocks. (vinodkv)
  3999. MAPREDUCE-3228. Fixed MR AM to timeout RPCs to bad NodeManagers. (vinodkv
  4000. via acmurthy)
  4001. MAPREDUCE-3284. Moved JobQueueClient to hadoop-mapreduce-client-core.
  4002. (acmurthy)
  4003. MAPREDUCE-3282. bin/mapred job -list throws exception. (acmurthy via
  4004. mahadev)
  4005. MAPREDUCE-3186. User jobs are getting hanged if the Resource manager
  4006. process goes down and comes up while job is getting executed.
  4007. (Eric Payne via mahadev)
  4008. MAPREDUCE-3285. Tests on branch-0.23 failing (Siddharth Seth via mahadev)
  4009. MAPREDUCE-3258. Fixed AM & JobHistory web-ui to display counters properly.
  4010. (Siddharth Seth via acmurthy)
  4011. MAPREDUCE-3290. Fixed a NPE in ClientRMService. (acmurthy)
  4012. MAPREDUCE-3185. RM Web UI does not sort the columns in some cases.
  4013. (Jonathan Eagles via mahadev)
  4014. MAPREDUCE-3292. In secure mode job submission fails with Provider
  4015. org.apache.hadoop.mapreduce.security.token.JobTokenIndentifier$Renewer
  4016. not found. (mahadev)
  4017. MAPREDUCE-3296. Fixed the remaining nine FindBugs warnings. (vinodkv)
  4018. MAPREDUCE-2775. Fixed ResourceManager and NodeManager to force a
  4019. decommissioned node to shutdown. (Devaraj K via vinodkv)
  4020. MAPREDUCE-3304. Fixed intermittent test failure due to a race in
  4021. TestRMContainerAllocator#testBlackListedNodes. (Ravi Prakash via acmurthy)
  4022. MAPREDUCE-3306. Fixed a bug in NodeManager ApplicationImpl that was causing
  4023. NodeManager to crash. (vinodkv)
  4024. MAPREDUCE-3295. TestAMAuthorization failing on branch 0.23. (vinodkv via mahadev)
  4025. MAPREDUCE-3183. hadoop-assemblies/src/main/resources/assemblies/hadoop-mapreduce-dist.xml
  4026. missing license header. (Hitesh Shah via tucu).
  4027. MAPREDUCE-3003. Publish MR JARs to Maven snapshot repository. (tucu)
  4028. MAPREDUCE-3199. Fixed pom files to include correct log4j configuration for
  4029. tests. (vinodkv)
  4030. MAPREDUCE-3204. mvn site:site fails on MapReduce. (tucu)
  4031. MAPREDUCE-3248. Fixed log4j properties. (vinodkv via acmurthy)
  4032. MAPREDUCE-3256. Added authorization checks for the protocol between
  4033. NodeManager and ApplicationMaster. (vinodkv via acmurthy)
  4034. MAPREDUCE-3274. Fixed a race condition in MRAppMaster that was causing a
  4035. task-scheduling deadlock. (Robert Joseph Evans via vinodkv)
  4036. MAPREDUCE-3171 merge from trunk reverted changes from MAPREDUCE-2747 MAPREDUCE-3240.
  4037. MAPREDUCE-3313. Fixed initialization of ClusterMetrics which was failing
  4038. TestResourceTrackerService sometimes. (Hitesh Shah via vinodkv)
  4039. MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories
  4040. in distributed-cache. (Hitesh Shah via vinodkv)
  4041. MAPREDUCE-2696. Fixed NodeManager to cleanup logs in a thread when logs'
  4042. aggregation is not enabled. (Siddharth Seth via vinodkv)
  4043. MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle
  4044. a couple of events in failure states correctly. (Hitesh Shah and Siddharth
  4045. Seth via vinodkv)
  4046. MAPREDUCE-3035. Fixed MR JobHistory to ensure rack information is present.
  4047. (chakravarthy via acmurthy)
  4048. MAPREDUCE-3321. Disabled a few MR tests for 0.23. (Hitesh Shah via
  4049. acmurthy)
  4050. MAPREDUCE-3220. Fixed TestCombineOutputCollector. (Devaraj K via acmurthy)
  4051. MAPREDUCE-3103. Implement Job ACLs for MRAppMaster.
  4052. (mahadev)
  4053. MAPREDUCE-3241. [Rumen] Fix Rumen to ignore the AMStartedEvent. (amarrk)
  4054. MAPREDUCE-3166. [Rumen] Make Rumen use job history api instead of relying
  4055. on current history file name format. (Ravi Gummadi via amarrk)
  4056. MAPREDUCE-3157. [Rumen] Fix TraceBuilder to handle 0.20 history file
  4057. names also. (Ravi Gummadi via amarrk)
  4058. MAPREDUCE-3081. Fix vaidya startup script. (gkesavan via suhas).
  4059. MAPREDUCE-2764. Fix renewal of dfs delegation tokens. (Owen via jitendra)
  4060. MAPREDUCE-3192. Fix Javadoc warning in JobClient.java and Cluster.java.
  4061. (jitendra)
  4062. MAPREDUCE-3237. Move LocalJobRunner to hadoop-mapreduce-client-core.
  4063. (tomwhite via acmurthy)
  4064. MAPREDUCE-3316. Rebooted link is not working properly.
  4065. (Bhallamudi Venkata Siva Kamesh via mahadev)
  4066. MAPREDUCE-3317. Rumen TraceBuilder is emiting null as hostname.
  4067. (Ravi Gummadi via mahadev)
  4068. MAPREDUCE-3332. contrib/raid compile breaks due to changes in hdfs/protocol/datatransfer/
  4069. Sender#writeBlock related to checksum handling (Hitesh Shah via mahadev)
  4070. MAPREDUCE-3337. Added missing license headers. (acmurthy)
  4071. Release 0.22.1 - Unreleased
  4072. INCOMPATIBLE CHANGES
  4073. NEW FEATURES
  4074. IMPROVEMENTS
  4075. OPTIMIZATIONS
  4076. BUG FIXES
  4077. MAPREDUCE-3837. Job tracker is not able to recover jobs after crash.
  4078. (Mayank Bansal via shv)
  4079. Release 0.22.0 - 2011-11-29
  4080. INCOMPATIBLE CHANGES
  4081. MAPREDUCE-1866. Removes deprecated class
  4082. org.apache.hadoop.streaming.UTF8ByteArrayUtils. (amareshwari)
  4083. MAPREDUCE-1664. Changes the behaviour of the combination of job-acls
  4084. when they function together with queue-acls. (Ravi Gummadi via vinodkv)
  4085. MAPREDUCE-2994. Fixed a bug in ApplicationID parsing that affects RM
  4086. UI. (Devaraj K via vinodkv)
  4087. MAPREDUCE-1788. o.a.h.mapreduce.Job shouldn't make a copy of the JobConf.
  4088. (Arun Murthy via mahadev)
  4089. NEW FEATURES
  4090. MAPREDUCE-1804. Stress-test tool for HDFS introduced in HDFS-708.
  4091. (Joshua Harlow via shv)
  4092. MAPREDUCE-220. Collect cpu and memory statistics per task. (Scott Chen via
  4093. acmurthy)
  4094. MAPREDUCE-1970. Reed-Solomon code implementation for HDFS RAID.
  4095. (Scott Chen via dhruba)
  4096. MAPREDUCE-2169. Integrated Reed-Solomon code with RaidNode. (Ramkumar
  4097. Vadali via schen)
  4098. MAPREDUCE-2936. Contrib Raid compilation broken after HDFS-1620. (vinodkv)
  4099. IMPROVEMENTS
  4100. MAPREDUCE-2141. Add an "extra data" field to Task for use by Mesos. (matei)
  4101. MAPREDUCE-2140. Regenerate fair scheduler design doc PDF. (matei)
  4102. MAPREDUCE-1546. Redirect all job pages to corresponding history page
  4103. if job is not in JT memory. (Scott Chen via sharad)
  4104. MAPREDUCE-1092. Enable assertions for unit tests. (Eli Collins via
  4105. cdouglas)
  4106. MAPREDUCE-1680. Add a metric recording JobTracker heartbeats processed.
  4107. (Dick King via cdouglas)
  4108. MAPREDUCE-1761. FairScheduler allows separate configuration of node
  4109. and rack locality wait time (Scott Chen via dhruba)
  4110. MAPREDUCE-1539. authorization checks for inter-server protocol
  4111. (based on HADOOP-6600) (Boris Shkolnik via shv)
  4112. MAPREDUCE-1798. Names the configuration keys for the Kerberos
  4113. principals better. (Boris Shkolnik via ddas)
  4114. MAPREDUCE-1773. streaming doesn't support jobclient.output.filter.
  4115. (Amareshwari Sriramadasu via vinodkv)
  4116. MAPREDUCE-1785. Add streaming config option for not emitting the key.
  4117. (Eli Collins via sharad)
  4118. MAPREDUCE-572. If #link is missing from uri format of -cacheArchive
  4119. then streaming does not throw error. (Amareshwari Sriramadasu via
  4120. vinodkv)
  4121. MAPREDUCE-1545. Add timestamps for first task type launched in job summary.
  4122. (Luke Lu via cdouglas)
  4123. MAPREDUCE-1543. Add an audit log for authentication events. (Amar Kamat and
  4124. Luke Lu via cdouglas)
  4125. MAPREDUCE-1762. Add ability to set values of task counters. (Scott Chen via
  4126. cdouglas)
  4127. MAPREDUCE-1533. Reduce overhead of logging and string manipulation during
  4128. heartbeat processing. (Amar Kamat and Dick King via cdouglas)
  4129. MAPREDUCE-1516. JobTracker issues delegation tokens only if the user's
  4130. authentication is Kerberos. (Jitendra Pandey via ddas)
  4131. MAPREDUCE-647. Update distcp forrest documentation to reflect the changes
  4132. of HADOOP-5472, MAPREDUCE-642 and HADOOP-5620. (Rodrigo Schmidt via
  4133. szetszwo)
  4134. MAPREDUCE-1851. Documents configuration parameters in streaming.
  4135. (amareshwari)
  4136. MAPREDUCE-1868. Add a read and connection timeout to JobClient while
  4137. pulling tasklogs. (Krishna Ramachandran via acmurthy)
  4138. MAPREDUCE-1778. Ensure failure to setup CompletedJobStatusStore is not
  4139. silently ignored by the JobTracker. (Krishna Ramachandran via acmurthy)
  4140. MAPREDUCE-1850. Includes job submit host information (name and ip) in
  4141. jobconf and jobdetails display (Krishna Ramachandran via amareshwari)
  4142. MAPREDUCE-1893. Slive with multiple reducers. (shv)
  4143. MAPREDUCE-1248. Fixes redudant memory copying in StreamKeyValUtil.
  4144. (Ruibang He via amareshwari)
  4145. MAPREDUCE-1840. Enhancements to Gridmix benchmark simulating user
  4146. diversity, queue replay, and task duration for JobTracker load testing.
  4147. Also includes compatibility with security enhancements, and scalability
  4148. improvements. (Amar Kamat, Rahul Singh, Hong Tang, and cdouglas)
  4149. MAPREDUCE-1848. Put number of speculative, data local, rack local
  4150. tasks in JobTracker metrics. (Scott Chen via dhruba)
  4151. MAPREDUCE-1935. Makes the Distcp to work in a secure environment.
  4152. (Boris Shkolnik via ddas)
  4153. MAPREDUCE-1945. The MapReduce component for HADOOP-6632.
  4154. (Kan Zhang & Jitendra Pandey via ddas)
  4155. MAPREDUCE-1936. Modify Gridmix3 to support more tunable parameters for
  4156. stress submission and sleep jobs. (Hong Tang via cdouglas)
  4157. MAPREDUCE-1733. Makes pipes applications secure. (Jitendra Pandey via ddas)
  4158. MAPREDUCE-1566. Adds a configuration attribute using which job clients can
  4159. specify a credentials file. The tokens from there will be passed to the job.
  4160. (Jitendra Pandey and Owen O'Malley via ddas)
  4161. MAPREDUCE-1624. Documents the job credentials and associated details to do
  4162. with delegation tokens (on the client side).
  4163. (Jitendra Pandey and Devaraj Das via ddas)
  4164. MAPREDUCE-1834. TestSimulatorDeterministicReplay timesout on trunk.
  4165. (Hong Tang via mahadev)
  4166. MAPREDUCE-1993. Fixes test failure
  4167. TestTrackerDistributedCacheManagerWithLinuxTaskController. (Devaraj Das
  4168. via amareshwari)
  4169. MAPREDUCE-1523. Making Mumak work with Capacity-Scheduler (Anirban Das
  4170. via mahadev)
  4171. MAPREDUCE-1920. Enables completed jobstatus store by default. (Tom White
  4172. via amareshwari)
  4173. MAPREDUCE-1881. Improve TaskTrackerInstrumentation to enable collection of
  4174. advanced metrics. (Matei Zaharia via acmurthy)
  4175. MAPREDUCE-1548. Hadoop archives preserve times and other properties from
  4176. original files. (Rodrigo Schmidt via dhruba)
  4177. MAPREDUCE-1517. Supports streaming job to run in the background. (Bochun Bai
  4178. via amareshwari)
  4179. MAPREDUCE-1819. RaidNode is now smarter in submitting Raid jobs. (Ramkumar
  4180. Vadali via schen)
  4181. MAPREDUCE-2132. A command line option in RaidShell to fix blocks using raid
  4182. MAPREDUCE-2147. Remove redundant lines in JobInProgress ctor.
  4183. (Harsh J Chouraria via cutting)
  4184. HADOOP-7007. Update the hudson-test-patch ant target to work with the
  4185. latest test-patch.sh script (gkesavan)
  4186. MAPREDUCE-1818. RaidNode can specify scheduling parameters. (Ramkumar
  4187. Vadali via schen)
  4188. MAPREDUCE-2051. Contribute a fair scheduler preemption system test.
  4189. (Todd Lipcon via tomwhite)
  4190. MAPREDUCE-1892. RaidNode can allow layered policies more efficiently.
  4191. (Ramkumar Vadali via schen)
  4192. MAPREDUCE-1592. Generate Eclipse's .classpath file from Ivy config.
  4193. (tomwhite via nigel)
  4194. MAPREDUCE-2073. TestTrackerDistributedCacheManager should be up-front
  4195. about requirements on build environment. (Todd Lipcon via tomwhite)
  4196. MAPREDUCE-2093. Herriot JT and TT clients should vend statistics. (cos)
  4197. MAPREDUCE-2167. Faster directory traversal for raid node. (Ramkumar Vadali
  4198. via schen)
  4199. MAPREDUCE-1931. Gridmix forrest documentation . (Ranjit Mathew via vinodkv).
  4200. MAPREDUCE-2184. Port DistRaid.java to new mapreduce API. (Ramkumar Vadali
  4201. via schen)
  4202. MAPREDUCE-1878. Add MRUnit documentation. (Aaron Kimball via tomwhite)
  4203. MAPREDUCE-2180. Add coverage of fair scheduler servlet to system test (todd)
  4204. MAPREDUCE-2250. Fix logging in raid code. (Ramkumar Vadali via schen)
  4205. MAPREDUCE-2260. Remove auto-generated native build files. (rvs via eli)
  4206. MAPREDUCE-2314. configure files that are generated as part of the released
  4207. tarball need to have executable bit set (rvs via cos)
  4208. MAPREDUCE-1159. Limit Job name in web UI to be 80 char long. (Harsh J
  4209. Chouraria via szetszwo)
  4210. MAPREDUCE-2337. Remove dependence of public MapReduce API on classes in
  4211. server package. (tomwhite)
  4212. MAPREDUCE-2383. Improve documentation of DistributedCache methods (Harsh J
  4213. Chouraria via todd)
  4214. MAPREDUCE-2222. Ivy resolve force mode should be turned off by default.
  4215. (Luke Lu via tomwhite)
  4216. MAPREDUCE-2103. task-controller shouldn't require o-r permissions.
  4217. (todd via eli)
  4218. MAPREDUCE-2505. Explain how to use ACLs in the fair scheduler.
  4219. (matei via eli)
  4220. MAPREDUCE-3138. Add a utility to help applications bridge changes in
  4221. Context Objects APIs due to MAPREDUCE-954. (omalley via acmurthy)
  4222. OPTIMIZATIONS
  4223. MAPREDUCE-1354. Enhancements to JobTracker for better performance and
  4224. scalability. (Arun C. Murthy & Richard King via acmurthy)
  4225. MAPREDUCE-1829. JobInProgress.findSpeculativeTask should use min() to
  4226. find the candidate instead of sort(). (Scott Chen via vinodkv)
  4227. BUG FIXES
  4228. MAPREDUCE-1845. FairScheduler.tasksToPreempt() can return negative number.
  4229. (Scott Chen via matei)
  4230. MAPREDUCE-1707. TaskRunner can get NPE in getting ugi from TaskTracker.
  4231. (Vinod Kumar Vavilapalli)
  4232. MAPREDUCE-1532. Ensures that delegation tokens is obtained as the
  4233. actual user when the proxy-user is used for submitting jobs. Also
  4234. refactors the DelegationTokenToRenew class. (ddas)
  4235. MAPREDUCE-1558. Fixes MRAdmin to look up the principal of the
  4236. JobTracker and use that in the RefreshUserToGroupsMapping protocol and
  4237. RefreshAuthorizationPolicyProtocol. (Boris Shkolnik via ddas)
  4238. MAPREDUCE-1662. Remove unused methods from TaskRunner. (Amareshwari
  4239. Sriramadasu via cdouglas)
  4240. MAPREDUCE-1617. Use IPv4 stack for unit tests. (Amar Kamat and Luke Lu via
  4241. cdouglas)
  4242. MAPREDUCE-1599. Fixes MRBench so that it reuses tokens across jobs
  4243. correctly. (Jitendra Nath Pandey via ddas)
  4244. MAPREDUCE-1836. Refresh for proxy superuser config (mr part for HDFS-1096).
  4245. (Boris Shkolnik via shv)
  4246. MAPREDUCE-1505. Create RPC client on job submission, not in cstr of Job
  4247. instance. (Dick King via cdouglas)
  4248. MAPREDUCE-1813. NPE in PipeMapred.MRErrorThread. (Ravi Gummadi via vinodkv)
  4249. MAPREDUCE-1225. Fixes DistributedCache to check if the file is fresh or not,
  4250. for the first localization also. (Zhong Wang via amareshwari)
  4251. MAPREDUCE-1559. Fixes the token renewer to use the JobTracker's
  4252. credentials for talking to the NameNode. (ddas)
  4253. MAPREDUCE-1492. Delete obsolete har files used on the parity files
  4254. of hdfs raid. (Rodrigo Schmidt via dhruba)
  4255. MAPREDUCE-1857. Removes unused configuration parameters in streaming.
  4256. (amareshwari)
  4257. MAPREDUCE-1887. MRAsyncDiskService now properly absolutizes volume root
  4258. paths. (Aaron Kimball via zshao)
  4259. MAPREDUCE-1863. Fix NPE in Rumen when processing null CDF for failed task
  4260. attempts. (Amar Kamat via cdouglas)
  4261. MAPREDUCE-1864. Removes uninitialized/unused variables in
  4262. org.apache.hadoop.streaming.PipeMapRed. (amareshwari)
  4263. MAPREDUCE-1888. Fixes Streaming to override output key and value types,
  4264. only if mapper/reducer is a command. (Ravi Gummadi via amareshwari)
  4265. MAPREDUCE-577. Fixes duplicate records in StreamXmlRecordReader.
  4266. (Ravi Gummadi via amareshwari)
  4267. MAPREDUCE-1894. Fixed a bug in DistributedRaidFileSystem.readFully()
  4268. that was causing it to loop infinitely. (Ramkumar Vadali via dhruba)
  4269. MAPREDUCE-1838. Reduce the time needed for raiding a bunch of files
  4270. by randomly assigning files to map tasks. (Ramkumar Vadali via dhruba)
  4271. MAPREDUCE-1820. Fix InputSampler to clone sampled keys. (Alex Kozlov via
  4272. cdouglas)
  4273. MAPREDUCE-1528. Incorporates the changes to the credentials API done in
  4274. HADOOP-6845. Also, introduces Credentials in JobConf, and in JobContext.
  4275. (Jitendra Pandey and Arun Murthy via ddas)
  4276. MAPREDUCE-1865. Rumen should also support jobhistory files generated using
  4277. trunk. (Amar Kamat via amareshwari)
  4278. MAPREDUCE-1621. Fixes NPE in TextOutputReader.getLastOutput if it has never
  4279. read any output. (amareshwari)
  4280. MAPREDUCE-1911. Fixes errors in -info message in streaming. (amareshwari)
  4281. MAPREDUCE-1772. Corrects errors in streaming documentation in forrest.
  4282. (amareshwari)
  4283. MAPREDUCE-1925. Fix failing TestRumenJobTraces. (Ravi Gummadi via cdouglas)
  4284. MAPREDUCE-1718. Fixes a bug in the construction of jobconf key for the
  4285. mapping that the tasks use at runtime for looking up delegation tokens.
  4286. (Boris Shkolnik via ddas)
  4287. MAPREDUCE-1701. Fixes a problem to do with exception handling in
  4288. delegation-token renewals. (Boris Shkolnik via ddas)
  4289. MAPREDUCE-1686. Fixes StreamUtil.goodClassOrNull to find classes without
  4290. package names. (Paul Burkhardt via amareshwari)
  4291. MAPREDUCE-1288. Fixes TrackerDistributedCacheManager to take into account
  4292. the owner of the localized file in the mapping from cache URIs to
  4293. CacheStatus objects. (ddas)
  4294. MAPREDUCE-1982. Fixes Rumen's TraceBuilder to extract job name from either
  4295. of configuration properties "mapreduce.job.name" and "mapred.job.name".
  4296. (Ravi Gummadi via amareshwari)
  4297. MAPREDUCE-1958. The MapReduce part corresponding to the HADOOP-6873.
  4298. (Boris Shkolnik & Owen O'Malley via ddas)
  4299. MAPREDUCE-1900. TaskTracker and JobTracker closes FileSystems, opened on
  4300. behalf of users that it no longer requires. (Kan Zhang and ddas via ddas)
  4301. MAPREDUCE-1992. Fixes a problem to do with bringing up the JobTracker in
  4302. unsecure mode. (Kan Zhang via ddas)
  4303. MAPREDUCE-1999. Fixes ClientProtocol to use the correct
  4304. DelegationTokenSelector. (Jitendra Pandey via ddas)
  4305. MAPREDUCE-1780. AccessControlList.toString() is used for serialization of
  4306. ACL in JobStatus.java. (Ravi Gummadi via vinodkv)
  4307. MAPREDUCE-1961. Fix ConcurrentModificationException in Gridmix during
  4308. shutdown. (Hong Tang via cdouglas)
  4309. MAPREDUCE-2000. Fix parsing of JobHistory lines in Rumen when quotes are
  4310. escaped. (Hong Tang via cdouglas)
  4311. MAPREDUCE-2022. Fixes compilation errors in TestSubmitJob. (amareshwari)
  4312. MAPREDUCE-1670. RAID policies should not scan their own destination path.
  4313. (Ramkumar Vadali via dhruba)
  4314. MAPREDUCE-1668. RaidNode Hars a directory only if all its parity files
  4315. have been created. (Ramkumar Vadali via dhruba)
  4316. MAPREDUCE-2021. Fixes duplicate hostnames in CombineFileInputFormat's
  4317. split locations. (amareshwari)
  4318. MAPREDUCE-1375. Fixes flaky test TestFileArgs. (Todd Lipcon via
  4319. amareshwari)
  4320. MAPREDUCE-2023. TestDFSIO should not stop reading if curSize != bufferSize.
  4321. (Hong Tang via szetszwo)
  4322. MAPREDUCE-2031. Fixes test failures TestTaskLauncher and
  4323. TestTaskTrackerLocalization. (Ravi Gummadi via amareshwari)
  4324. MAPREDUCE-2046. Fixes CombineFileInputFormat to allow splits with size
  4325. less than DFS block size. (dhruba borthakur via amareshwari)
  4326. MAPREDUCE-1975. Fixes unnecessary InterruptedException log in gridmix.
  4327. (Ravi Gummadi via amareshwari)
  4328. MAPREDUCE-1597. Fixes CombineFileInputFormat to work with non-splittable
  4329. files. (amareshwari)
  4330. MAPREDUCE-2032. Fixes TestJobCleanup to cleanup test directory in
  4331. tearDown. (Dick King via amareshwari)
  4332. MAPREDUCE-1979. Fixes "Output directory already exists" error in gridmix
  4333. when gridmix.output.directory is not defined. (Ravi Gummadi via
  4334. amareshwari)
  4335. MAPREDUCE-1918. Adds documentation to Rumen. (Amar Kamat via amareshwari)
  4336. MAPREDUCE-2078. Fixes TraceBuilder to generate traces when a globbed job
  4337. history path is given. (Amar Kamat via amareshwari)
  4338. MAPREDUCE-1989. Fixes error message in gridmix when user resolver is set
  4339. and no user list is given. (Ravi Gummadi via amareshwari)
  4340. MAPREDUCE-2067. Distinct minicluster services (e.g. NN and JT) overwrite
  4341. each other's service policies. (Aaron T. Myers via tomwhite)
  4342. MAPREDUCE-2029. DistributedRaidFileSystem removes itself from FileSystem
  4343. cache when it is closed. (Ramkumar Vadali via dhruba)
  4344. MAPREDUCE-1816. HAR files used for RAID parity-bite have configurable
  4345. partfile size. (Ramkumar Vadali via dhruba)
  4346. MAPREDUCE-2082. Fixes Pipes to create the jobtoken file in the right
  4347. place. (Jitendra Pandey via ddas)
  4348. MAPREDUCE-2095. Fixes Gridmix to run from compressed traces. (Ranjit
  4349. Mathew via amareshwari)
  4350. MAPREDUCE-1908. DistributedRaidFileSystem now handles ChecksumException
  4351. correctly. (Ramkumar Vadali via schen)
  4352. MAPREDUCE-2126. JobQueueJobInProgressListener's javadoc is inconsistent
  4353. with source code. (Jingguo Yao via tomwhite)
  4354. MAPREDUCE-2143. HarFileSystem is able to handle spaces in pathnames.
  4355. (Ramkumar Vadali via dhruba)
  4356. MAPREDUCE-1867. Remove unused methods in
  4357. org.apache.hadoop.streaming.StreamUtil. (amareshwari via tomwhite)
  4358. MAPREDUCE-2146. Raid does not affect access time of a source file.
  4359. (Ramkumar Vadali via dhruba)
  4360. MAPREDUCE-2150. RaidNode periodically fixes corrupt blocks. (Ramkumar Vadali via
  4361. schen)
  4362. MAPREDUCE-2099. RaidNode recreates outdated parity HARs. (Ramkumar Vadali
  4363. via schen)
  4364. MAPREDUCE-2173. Fix race condition in TestBlockFixer that was
  4365. causing intermittent failure (Patrick Kling via dhruba)
  4366. MAPREDUCE-2142. Refactor RaidNode so that the map-reduce component is
  4367. clearly separated out. (Patrick Kling via dhruba)
  4368. MAPREDUCE-2179. Fix RaidBlockSender compilation failure. (Ramkumar Vadali
  4369. via schen)
  4370. MAPREDUCE-2034. TestSubmitJob triggers NPE instead of permissions error.
  4371. (Todd Lipcon via tomwhite)
  4372. MAPREDUCE-2195. New property for local conf directory in
  4373. system-test-mapreduce.xml file. (cos)
  4374. MAPREDUCE-1783. FairScheduler initializes tasks only when the job can be
  4375. run. (Ramkumar Vadali via schen)
  4376. MAPREDUCE-2224. Fix synchronization bugs in JvmManager. (todd)
  4377. MAPREDUCE-714. JobConf.findContainingJar unescapes unnecessarily on linux (todd)
  4378. MAPREDUCE-2096. Secure local filesystem IO from symlink vulnerabilities (todd)
  4379. MAPREDUCE-2234. If Localizer can't create task log directory, it should fail
  4380. on the spot. (todd)
  4381. MAPREDUCE-2219. JobTracker should not try to remove mapred.system.dir
  4382. during startup. (todd)
  4383. MAPREDUCE-2207. Task-cleanup task should not be scheduled on the node that
  4384. the task just failed. (Liyin Liang via schen)
  4385. MAPREDUCE-2084. Remove deprecate annotation for package file. The package
  4386. classes themselves are already deprecated. This removes an Eclipse error.
  4387. (tomwhite via nigel)
  4388. MAPREDUCE-2248. DistributedRaidFileSystem should unraid only the corrupt
  4389. block (Ramkumar Vadali via schen)
  4390. MAPREDUCE-1085. For tasks, "ulimit -v -1" is being run when user doesn't
  4391. specify a ulimit (todd)
  4392. MAPREDUCE-2282. Fix TestMRServerPorts for the changes in
  4393. TestHDFSServerPorts. (shv via szetszwo)
  4394. MAPREDUCE-2238. Fix permissions handling to avoid leaving undeletable
  4395. directories in local dirs. (todd)
  4396. MAPREDUCE-2277. TestCapacitySchedulerWithJobTracker needs to wait for jobs
  4397. to complete before testing status. (todd)
  4398. MAPREDUCE-2253. Servlets should specify content type (todd)
  4399. MAPREDUCE-2283. Add timeout for Raid Tests (Ramkumar Vadali via schen)
  4400. MAPREDUCE-1754. Replace mapred.persmissions.supergroup with an
  4401. acl : mapreduce.cluster.administrators (Amareshwari Sriramadasu via shv)
  4402. MAPREDUCE-2256. FairScheduler fairshare preemption from multiple pools may
  4403. preempt all tasks from one pool causing that pool to go below fairshare.
  4404. (Priyo Mustafi via shv)
  4405. MAPREDUCE-2281. MR part of HADOOP-6642. (Chris Douglas, Po Cheung via shv)
  4406. MAPREDUCE-2200. TestUmbilicalProtocolWithJobToken is failing without Krb
  4407. evironment: needs to be conditional. (cos)
  4408. MAPREDUCE-2077. Resolve name clash in the deprecated
  4409. o.a.h.util.MemoryCalculatorPlugin (Luke Lu via shv)
  4410. MAPREDUCE-2188. The new API MultithreadedMapper doesn't initialize
  4411. RecordReader. (Owen O'Malley via shv)
  4412. MAPREDUCE-1915. Fix IndexOutOfBoundsException in IndexCache.
  4413. (Priyo Mustafi via shv)
  4414. MAPREDUCE-1974. Fix multiple preemtions of the same task in FairScheduler.
  4415. (Scott Chen via shv)
  4416. MAPREDUCE-2304. Fix TestMRCLI to allow hostname with a hyphen (-).
  4417. (Priyo Mustafi via shv)
  4418. MAPREDUCE-1825. jobqueue_details.jsp and FairSchedulerServelet should not
  4419. call finishedMaps and finishedReduces when job is not initialized.
  4420. (Scott Chen via shv)
  4421. MAPREDUCE-2285. MiniMRCluster does not start after ant test-patch
  4422. (todd)
  4423. MAPREDUCE-2315. javadoc is failing in nightly build (todd)
  4424. MAPREDUCE-2054. Hierarchical queue implementation broke dynamic queue
  4425. addition in Dynamic Scheduler. (Thomas Sandholm via tomwhite)
  4426. MAPREDUCE-2272. Job ACL file should not be executable
  4427. (Harsh J Chouraria via todd)
  4428. MAPREDUCE-2241. ClusterWithLinuxTaskController should accept relative path
  4429. on the command line. (todd)
  4430. MAPREDUCE-2251. Remove unused mapreduce.job.userhistorylocation config.
  4431. (Harsh J Chouraria via todd)
  4432. MAPREDUCE-2284. TestLocalRunner.testMultiMaps times out (todd)
  4433. MAPREDUCE-2336. Tool-related packages should be in the Tool javadoc group.
  4434. (tomwhite)
  4435. MAPREDUCE-2394. JUnit output format doesn't propagate into raid contrib
  4436. build. (todd)
  4437. MAPREDUCE-2392. TaskTracker shutdown in the tests sometimes take 60s.
  4438. (tomwhite)
  4439. MAPREDUCE-2437. SLive uses only part* files to generating the final report.
  4440. (shv)
  4441. MAPREDUCE-2428. start-mapred.sh script fails if HADOOP_HOME is not set.
  4442. (tomwhite via eli)
  4443. MAPREDUCE-2445. Fix TestMiniMRWithDFSWithDistinctUsers to be a valid test.
  4444. (todd)
  4445. MAPREDUCE-2457. Job submission should inject group.name on the JobTracker
  4446. (Alejandro Abdelnur via todd)
  4447. MAPREDUCE-2472. Extra whitespace in mapred.child.java.opts breaks JVM
  4448. initialization. (Aaron T. Myers via todd)
  4449. MAPREDUCE-2222. Ivy resolve force mode should be turned off by default.
  4450. (Luke Lu via tomwhite)
  4451. MAPREDUCE-2486. Incorrect snapshot dependency published in .pom files
  4452. (todd)
  4453. MAPREDUCE-2327. MapTask doesn't need to put username information in
  4454. SpillRecord. (todd via tomwhite)
  4455. MAPREDUCE-2515. MapReduce code references some deprecated options
  4456. (Ari Rabkin via todd)
  4457. MAPREDUCE-2487. ChainReducer uses MAPPER_BY_VALUE instead of
  4458. REDUCER_BY_VALUE. (Devaraj K via todd)
  4459. MAPREDUCE-2185. Fix infinite loop at creating splits using
  4460. CombineFileInputFormat. (Ramkumar Vadali via schen)
  4461. MAPREDUCE-2571. CombineFileInputFormat.getSplits throws a
  4462. java.lang.ArrayStoreException. (Bochun Bai via todd)
  4463. MAPREDUCE-2767. Remove Linux task-controller. (Milind Bhandarkar via shv)
  4464. MAPREDUCE-2991. queueinfo.jsp fails to show queue status for Capacity
  4465. scheduler if queue names contain special symbols. (Priyo Mustafi via shv)
  4466. MAPREDUCE-2531. Fixed jobcontrol to downgrade JobID. (Robert Evans via
  4467. acmurthy)
  4468. MAPREDUCE-3139. SlivePartitioner generates negative partitions. (jghoman)
  4469. Release 0.21.1 - Unreleased
  4470. NEW FEATURES
  4471. MAPREDUCE-2040. Forrest Documentation for Dynamic Priority Scheduler.
  4472. (Thomas Sandholm via tomwhite)
  4473. BUG FIXES
  4474. MAPREDUCE-1897. trunk build broken on compile-mapred-test (cos)
  4475. MAPREDUCE-1280. Update Eclipse plugin to the new eclipse.jdt API.
  4476. (Alex Kozlov via szetszwo)
  4477. MAPREDUCE-1984. herriot TestCluster fails because exclusion is not there
  4478. (Balaji Rajagopalan via cos)
  4479. MAPREDUCE-2090. Clover build doesn't generate per-test coverage. (cos)
  4480. MAPREDUCE-2134. ant binary-system is broken in mapreduce project. (cos)
  4481. MAPREDUCE-1905. Fixes Context.setStatus() and progress() apis.
  4482. (amareshwari)
  4483. MAPREDUCE-1809. Ant build changes for Streaming system tests in contrib
  4484. projects. (Vinay Kumar Thota via amareshwari)
  4485. MAPREDUCE-2223. TestMRCLI might fail on Ubuntu with default /etc/hosts
  4486. (cos)
  4487. MAPREDUCE-2228. Remove java5 dependencies from build. (cos)
  4488. MAPREDUCE-1929. Allow artifacts to be published to the staging Apache Nexus
  4489. Maven Repository. (tomwhite)
  4490. MAPREDUCE-2317. Fix a NPE in HadoopArchives. (Devaraj K via szetszwo)
  4491. MAPREDUCE-2127. mapreduce trunk builds are filing on hudson.
  4492. (Bruno Mahé via eli)
  4493. MAPREDUCE-2779. JobSplitWriter.java can't handle large job.split file.
  4494. (Ming Ma via shv)
  4495. Release 0.21.0 - 2010-08-13
  4496. INCOMPATIBLE CHANGES
  4497. MAPREDUCE-516. Fix the starvation problem in the Capacity Scheduler
  4498. when running High RAM Jobs. (Arun Murthy via yhemanth)
  4499. MAPREDUCE-358. Change org.apache.hadoop.examples. AggregateWordCount
  4500. and org.apache.hadoop.examples.AggregateWordHistogram to use new
  4501. mapreduce api. (Amareshwari Sriramadasu via sharad)
  4502. MAPREDUCE-245. Change Job and jobcontrol classes to use the List interface
  4503. rather than ArrayList in APIs. (Tom White via cdouglas)
  4504. MAPREDUCE-766. Enhanced list-blacklisted-trackers to display reasons
  4505. for blacklisting a node. (Sreekanth Ramakrishnan via yhemanth)
  4506. MAPREDUCE-817. Add a cache for retired jobs with minimal job info and
  4507. provide a way to access history file url. (sharad)
  4508. MAPREDUCE-711. Moved Distributed Cache from Common to Map/Reduce
  4509. project. (Vinod Kumar Vavilapalli via yhemanth)
  4510. MAPREDUCE-895. Per the contract elucidated in HADOOP-6201, throw
  4511. FileNotFoundException from FileSystem::listStatus rather than returning
  4512. null. (Jakob Homan via cdouglas)
  4513. MAPREDUCE-479. Provide full task id to map output servlet rather than the
  4514. reduce id, only. (Jiaqi Tan via cdouglas)
  4515. MAPREDUCE-873. Simplify job recovery. Incomplete jobs are resubmitted on
  4516. jobtracker restart. Removes a public constructor in JobInProgress. (sharad)
  4517. HADOOP-6230. Moved process tree and memory calculator related classes from
  4518. Common to Map/Reduce. (Vinod Kumar Vavilapalli via yhemanth)
  4519. MAPREDUCE-157. Refactor job history APIs and change the history format to
  4520. JSON. (Jothi Padmanabhan via sharad)
  4521. MAPREDUCE-849. Rename configuration properties. (Amareshwari Sriramadasu
  4522. via sharad)
  4523. MAPREDUCE-1287. Only call the partitioner with more than one reducer.
  4524. (cdouglas)
  4525. MAPREDUCE-1385. Use the new UserGroupInformation from HADOOP-6299.
  4526. (ddas via omalley)
  4527. MAPREDUCE-1493. Authorization for job-history pages. (vinodkv)
  4528. MAPREDUCE-1607. Task controller may not set permissions for a
  4529. task cleanup attempt's log directory (Amareshwari Sriramadasu via vinodkv)
  4530. MAPREDUCE-1683. Remove JNI calls from ClusterStatus cstr. (Arun Murthy and
  4531. Luke Lu via cdouglas)
  4532. MAPREDUCE-1855. Makes the refresh methods (for groups and proxy users)
  4533. independent of the client side configuration. (Boris Shkolnik via ddas)
  4534. NEW FEATURES
  4535. MAPREDUCE-1774. Large-scale Automated Framework (Sharad Agarwal, Sreekanth
  4536. Ramakrishnan, Konstantin Boudnik, at all via cos)
  4537. MAPREDUCE-706. Support for FIFO pools in the fair scheduler.
  4538. (Matei Zaharia)
  4539. MAPREDUCE-546. Provide sample fair scheduler config file in conf/ and use
  4540. it by default if no other config file is specified. (Matei Zaharia)
  4541. MAPREDUCE-551. Preemption support in the Fair Scheduler. (Matei Zaharia)
  4542. MAPREDUCE-567. Add a new example MR that always fails. (Philip Zeyliger
  4543. via tomwhite)
  4544. MAPREDUCE-211. Provides ability to run a health check script on the
  4545. tasktracker nodes and blacklist nodes if they are unhealthy.
  4546. (Sreekanth Ramakrishnan via yhemanth)
  4547. MAPREDUCE-637. Add an example, distbbp, which able to compute the n th bit
  4548. of Pi for some large n. (szetszwo)
  4549. MAPREDUCE-532. Provide a way to limit the number of used slots
  4550. per queue in the capacity scheduler.
  4551. (Rahul Kumar Singh via yhemanth)
  4552. MAPREDUCE-467. Provide ability to collect statistics about total tasks
  4553. and succeeded tasks in different time windows. (sharad)
  4554. MAPREDUCE-740. Log a job-summary at the end of a job, while allowing it
  4555. to be configured to use a custom appender if desired. (acmurthy)
  4556. MAPREDUCE-814. Provide a way to configure completed job history files
  4557. to be on HDFS. (sharad)
  4558. MAPREDUCE-800. MRUnit should support the new API. (Aaron Kimball via
  4559. tomwhite)
  4560. MAPREDUCE-798. MRUnit should be able to test a succession of MapReduce
  4561. passes. (Aaron Kimball via tomwhite)
  4562. MAPREDUCE-768. Provide an option to dump jobtracker configuration in JSON
  4563. format to standard output. (V.V.Chaitanya Krishna via yhemanth)
  4564. MAPREDUCE-824. Add support for a hierarchy of queues in the capacity
  4565. scheduler. (Rahul Kumar Singh via yhemanth)
  4566. MAPREDUCE-751. Add Rumen, a tool for extracting statistics from job tracker
  4567. logs and generating job traces for simulation and analysis. (Dick King via
  4568. cdouglas)
  4569. MAPREDUCE-830. Add support for splittable compression to TextInputFormats.
  4570. (Abdul Qadeer via cdouglas)
  4571. MAPREDUCE-861. Add support for hierarchical queues in the Map/Reduce
  4572. framework. (Rahul Kumar Singh via yhemanth)
  4573. MAPREDUCE-776. Add Gridmix, a benchmark processing Rumen traces to simulate
  4574. a measured mix of jobs on a cluster. (cdouglas)
  4575. MAPREDUCE-862. Enhance JobTracker UI to display hierarchical queues.
  4576. (V.V.Chaitanya Krishna via yhemanth)
  4577. MAPREDUCE-777. Brand new apis to track and query jobs as a
  4578. replacement for JobClient. (Amareshwari Sriramadasu via acmurthy)
  4579. MAPREDUCE-775. Add native and streaming support for Vertica as an input
  4580. or output format taking advantage of parallel read and write properties of
  4581. the DBMS. (Omer Trajman via ddas)
  4582. MAPREDUCE-679. XML-based metrics as JSP servlet for JobTracker.
  4583. (Aaron Kimball via tomwhite)
  4584. MAPREDUCE-980. Modify JobHistory to use Avro for serialization. (cutting)
  4585. MAPREDUCE-728. Add Mumak, a Hadoop map/reduce simulator. (Arun C Murthy,
  4586. Tamas Sarlos, Anirban Dasgupta, Guanying Wang, and Hong Tang via cdouglas)
  4587. MAPREDUCE-1383. Automates fetching of delegation tokens in File*Formats
  4588. Distributed Cache and Distcp. Also, provides a config
  4589. mapreduce.job.hdfs-servers that the jobs can populate with a comma
  4590. separated list of namenodes. The job client automatically fetches
  4591. delegation tokens from those namenodes. (Boris Shkolnik via ddas)
  4592. MAPREDUCE-698. Per-pool task limits for the fair scheduler.
  4593. (Kevin Peterson via matei)
  4594. MAPREDUCE-1026. Does mutual authentication of the shuffle
  4595. transfers using a shared JobTracker generated key.
  4596. (Boris Shkolnik via ddas)
  4597. MAPREDUCE-744. Introduces the notion of a public distributed cache.
  4598. (Devaraj Das)
  4599. MAPREDUCE-1338. Introduces the notion of token cache using which
  4600. tokens and secrets can be sent by the Job client to the JobTracker.
  4601. (Boris Shkolnik via ddas)
  4602. HDFS-503. This patch implements an optional layer over HDFS that
  4603. implements offline erasure-coding. It can be used to reduce the
  4604. total storage requirements of HDFS. (dhruba)
  4605. MAPREDUCE-1432. Adds hooks in the jobtracker and tasktracker
  4606. for loading the tokens in the user's ugi. This is required
  4607. for the copying of files from the hdfs. (ddas)
  4608. MAPREDUCE-1335. Adds SASL Kerberos/Digest authentication in MapReduce.
  4609. (Kan Zhang via ddas)
  4610. MAPREDUCE-1464. Makes a compatible change in JobTokenIdentifier to
  4611. account for HADOOP-6510. (Jitendra Nath Pandey via ddas)
  4612. MAPREDUCE-1433. Add a delegation token for MapReduce. (omalley)
  4613. MAPREDUCE-1307. Introduces the Job level ACLs feature.
  4614. (Vinod Kumar Vavilapalli via ddas)
  4615. MAPREDUCE-1430. JobTracker automatically renews delegation tokens for jobs.
  4616. (Boris Shkolnik via ddas)
  4617. MAPREDUCE-1455. Introduces job-level authorization for mapreduce servlets.
  4618. (Ravi Gummadi via vinodkv)
  4619. IMPROVEMENTS
  4620. MAPREDUCE-463. Makes job setup and cleanup tasks as optional.
  4621. (Amareshwari Sriramadasu via sharad)
  4622. MAPREDUCE-502. Allow jobtracker to be configured with zero completed jobs
  4623. in memory. (Amar Kamat via sharad)
  4624. MAPREDUCE-416. Moves the history file to a "done" folder whenever a job
  4625. completes. (Amar Kamat via ddas)
  4626. MAPREDUCE-646. Increase srcfilelist replication number in dictcp job.
  4627. (Ravi Gummadi via szetszwo)
  4628. HADOOP-6106. Updated hadoop-core and test jars from hudson trunk
  4629. build #12. (Giridharan Kesavan)
  4630. MAPREDUCE-642. A option to distcp that allows preserving the full
  4631. source path of a file in the specified destination directory.
  4632. (Rodrigo Schmidt via dhruba)
  4633. MAPREDUCE-686. Move TestSpeculativeExecution.Fake* into a separate class
  4634. so that it can be used by other tests. (Jothi Padmanabhan via sharad)
  4635. MAPREDUCE-625. Modify TestTaskLimits to improve execution time.
  4636. (Jothi Padmanabhan via sharad)
  4637. MAPREDUCE-465. Deprecate o.a.h.mapred.lib.MultithreadedMapRunner and add
  4638. test for o.a.h.mapreduce.lib.MultithreadedMapper.
  4639. (Amareshwari Sriramadasu via sharad)
  4640. MAPREDUCE-701. Improves the runtime of the TestRackAwareTaskPlacement
  4641. by making it a unit test. (Jothi Padmanabhan via ddas)
  4642. MAPREDUCE-371. Change KeyFieldBasedComparator and KeyFieldBasedPartitioner
  4643. to use new api. (Amareshwari Sriramadasu via sharad)
  4644. MAPREDUCE-623. Resolve javac warnings in mapreduce. (Jothi Padmanabhan
  4645. via sharad)
  4646. MAPREDUCE-655. Change KeyValueLineRecordReader and KeyValueTextInputFormat
  4647. to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
  4648. MAPREDUCE-632. Merge TestCustomOutputCommitter with
  4649. TestCommandLineJobSubmission. (Jothi Padmanabhan via sharad)
  4650. MAPREDUCE-627. Improves execution time of TestTrackerBlacklistAcrossJobs.
  4651. (Jothi Padmanabhan via ddas)
  4652. MAPREDUCE-630. Improves execution time of TestKillCompletedJob.
  4653. (Jothi Padmanabhan via ddas)
  4654. MAPREDUCE-626. Improves the execution time of TestLostTracker.
  4655. (Jothi Padmanabhan via ddas)
  4656. MAPREDUCE-353. Makes the shuffle read and connection timeouts
  4657. configurable. (Ravi Gummadi via ddas)
  4658. MAPREDUCE-739. Allow relative paths to be created in archives. (Mahadev
  4659. Konar via cdouglas)
  4660. MAPREDUCE-772. Merge HADOOP-4010 changes to LineRecordReader into mapreduce
  4661. package. (Abdul Qadeer via cdouglas)
  4662. MAPREDUCE-785. Separate sub-test of TestReduceFetch to be included in
  4663. MR-670. (Jothi Padmanabhan via cdouglas)
  4664. MAPREDUCE-784. Modify TestUserDefinedCounters to use LocalJobRunner
  4665. instead of MiniMR. (Jothi Padmanabhan via sharad)
  4666. HADOOP-6160. Fix releaseaudit target to run on specific directories.
  4667. (gkesavan)
  4668. MAPREDUCE-782. Use PureJavaCrc32 in SpillRecord. (Todd Lipcon via
  4669. szetszwo)
  4670. MAPREDUCE-369. Change org.apache.hadoop.mapred.lib.MultipleInputs to
  4671. use new api. (Amareshwari Sriramadasu via sharad)
  4672. MAPREDUCE-373. Change org.apache.hadoop.mapred.lib.FieldSelectionMapReduce
  4673. to use new api. (Amareshwari Sriramadasu via sharad)
  4674. MAPREDUCE-628. Improves the execution time of TestJobInProgress.
  4675. (Jothi Padmanabhan via ddas)
  4676. MAPREDUCE-793. Creates a new test that consolidates a few tests to
  4677. include in the commit-test list. (Jothi Padmanabhan via ddas)
  4678. MAPREDUCE-797. Adds combiner support to MRUnit MapReduceDriver.
  4679. (Aaron Kimball via johan)
  4680. MAPREDUCE-656. Change org.apache.hadoop.mapred.SequenceFile* classes
  4681. to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
  4682. MAPREDUCE-670. Creates ant target for 10 mins patch test build.
  4683. (Jothi Padmanabhan via gkesavan)
  4684. MAPREDUCE-375. Change org.apache.hadoop.mapred.lib.NLineInputFormat
  4685. and org.apache.hadoop.mapred.MapFileOutputFormat to use new api.
  4686. (Amareshwari Sriramadasu via ddas)
  4687. MAPREDUCE-779. Added node health failure counts into
  4688. JobTrackerStatistics. (Sreekanth Ramakrishnan via yhemanth)
  4689. MAPREDUCE-842. Setup secure permissions for localized job files,
  4690. intermediate outputs and log files on tasktrackers.
  4691. (Vinod Kumar Vavilapalli via yhemanth)
  4692. MAPREDUCE-478. Allow map and reduce jvm parameters, environment variables
  4693. and ulimit to be set separately.
  4694. Configuration changes:
  4695. add mapred.map.child.java.opts
  4696. add mapred.reduce.child.java.opts
  4697. add mapred.map.child.env
  4698. add mapred.reduce.child.ulimit
  4699. add mapred.map.child.env
  4700. add mapred.reduce.child.ulimit
  4701. deprecated mapred.child.java.opts
  4702. deprecated mapred.child.env
  4703. deprecated mapred.child.ulimit
  4704. (acmurthy)
  4705. MAPREDUCE-767. Remove the dependence on the CLI 2.0 snapshot.
  4706. (Amar Kamat via omalley)
  4707. MAPREDUCE-712. Minor efficiency tweaks to RandomTextWriter. (cdouglas)
  4708. MAPREDUCE-870. Remove the job retire thread and the associated
  4709. config parameters. (sharad)
  4710. MAPREDUCE-874. Rename the PiEstimator example to QuasiMonteCarlo.
  4711. (szetszwo)
  4712. MAPREDUCE-336. Allow logging level of map/reduce tasks to be configurable.
  4713. Configuration changes:
  4714. add mapred.map.child.log.level
  4715. add mapred.reduce.child.log.level
  4716. (acmurthy)
  4717. MAPREDUCE-355. Update mapred.join package to use the new API. (Amareshwari
  4718. Sriramadasu via cdouglas)
  4719. HADOOP-6184. Updated hadoop common and test jars to get the new API
  4720. in Configuration for dumping in JSON format from Hudson trunk build #68.
  4721. (yhemanth)
  4722. MAPREDUCE-476. Extend DistributedCache to work locally (LocalJobRunner).
  4723. (Philip Zeyliger via tomwhite)
  4724. MAPREDUCE-825. JobClient completion poll interval of 5s causes slow tests
  4725. in local mode. (Aaron Kimball via tomwhite)
  4726. MAPREDUCE-910. Support counters in MRUnit. (Aaron Kimball via cdouglas)
  4727. MAPREDUCE-788. Update gridmix2 to use the new API (Amareshwari Sriramadasu
  4728. via cdouglas)
  4729. MAPREDUCE-875. Make DBRecordReader execute queries lazily. (Aaron Kimball
  4730. via enis)
  4731. MAPREDUCE-318. Modularizes the shuffle code. (Jothi Padmanabhan and
  4732. Arun Murthy via ddas)
  4733. MAPREDUCE-936. Allow a load difference for fairshare scheduler.
  4734. (Zheng Shao via dhruba)
  4735. MAPREDUCE-370. Update MultipleOutputs to use the API, merge funcitonality
  4736. of MultipleOutputFormat. (Amareshwari Sriramadasu via cdouglas)
  4737. MAPREDUCE-898. Changes DistributedCache to use the new API.
  4738. (Amareshwari Sriramadasu via ddas)
  4739. MAPREDUCE-144. Includes dump of the process tree in task diagnostics when
  4740. a task is killed due to exceeding memory limits.
  4741. (Vinod Kumar Vavilapalli via yhemanth)
  4742. MAPREDUCE-945. Modifies MRBench and TestMapRed to use ToolRunner so that
  4743. options such as queue name can be passed via command line.
  4744. (Sreekanth Ramakrishnan via yhemanth)
  4745. MAPREDUCE-963. Deprecate o.a.h.mapred.FileAlreadyExistsException and
  4746. replace it with o.a.h.fs.FileAlreadyExistsException. (Boris Shkolnik
  4747. via szetszwo)
  4748. MAPREDUCE-960. Remove an unnecessary intermediate copy and obsolete API
  4749. from KeyValueLineRecordReader. (cdouglas)
  4750. MAPREDUCE-930. Modify Rumen to resolve paths in the canonical way, rather
  4751. than defaulting to the local filesystem. (cdouglas)
  4752. MAPREDUCE-944. Extend the LoadManager API of the fair-share scheduler
  4753. to support regulating tasks for a job based on resources currently in use
  4754. by that job. (dhruba)
  4755. MAPREDUCE-973. Move FailJob and SleepJob from examples to test. (cdouglas
  4756. via omalley)
  4757. MAPREDUCE-966. Modify Rumen to clean up interfaces and simplify integration
  4758. with other tools. (Hong Tang via cdouglas)
  4759. MAPREDUCE-856. Setup secure permissions for distributed cache files.
  4760. (Vinod Kumar Vavilapalli via yhemanth)
  4761. MAPREDUCE-885. More efficient SQL queries for DBInputFormat. (Aaron Kimball
  4762. via enis)
  4763. MAPREDUCE-284. Enables ipc.client.tcpnodelay in Tasktracker's Child.
  4764. (Ravi Gummadi via sharad)
  4765. MAPREDUCE-916. Split the documentation to match the project split.
  4766. (Corinne Chandel via omalley)
  4767. MAPREDUCE-649. Validate a copy by comparing the source and destination
  4768. checksums in distcp. Also adds an intra-task retry mechanism for errors
  4769. detected during the copy. (Ravi Gummadi via cdouglas)
  4770. MAPREDUCE-654. Add a -dryrun option to distcp printing a summary of the
  4771. file data to be copied, without actually performing the copy. (Ravi Gummadi
  4772. via cdouglas)
  4773. MAPREDUCE-664. Display the number of files deleted by distcp when the
  4774. -delete option is specified. (Ravi Gummadi via cdouglas)
  4775. MAPREDUCE-781. Let the name of distcp jobs be configurable. (Venkatesh S
  4776. via cdouglas)
  4777. MAPREDUCE-975. Add an API in job client to get the history file url for
  4778. a given job id. (sharad)
  4779. MAPREDUCE-905. Add Eclipse launch tasks for MapReduce. (Philip Zeyliger
  4780. via tomwhite)
  4781. MAPREDUCE-277. Makes job history counters available on the job history
  4782. viewers. (Jothi Padmanabhan via ddas)
  4783. MAPREDUCE-893. Provides an ability to refresh queue configuration
  4784. without restarting the JobTracker.
  4785. (Vinod Kumar Vavilapalli and Rahul Kumar Singh via yhemanth)
  4786. MAPREDUCE-1011. Add build.properties to svn and git ignore. (omalley)
  4787. MAPREDUCE-954. Change Map-Reduce context objects to be interfaces.
  4788. (acmurthy)
  4789. MAPREDUCE-639. Change Terasort example to reflect the 2009 updates.
  4790. (omalley)
  4791. MAPREDUCE-1063. Document gridmix benchmark. (cdouglas)
  4792. MAPREDUCE-931. Use built-in interpolation classes for making up task
  4793. runtimes in Rumen. (Dick King via cdouglas)
  4794. MAPREDUCE-1012. Mark Context interfaces as public evolving. (Tom White via
  4795. cdouglas)
  4796. MAPREDUCE-971. Document use of distcp when copying to s3, managing timeouts
  4797. in particular. (Aaron Kimball via cdouglas)
  4798. HDFS-663. DFSIO for append. (shv)
  4799. HDFS-641. Move all of the components that depend on map/reduce to
  4800. map/reduce. (omalley)
  4801. HADOOP-5107. Use Maven ant tasks to publish artifacts. (Giridharan Kesavan
  4802. via omalley)
  4803. MAPREDUCE-1229. Allow customization of job submission policy in Mumak.
  4804. (Hong Tang via cdouglas)
  4805. MAPREDUCE-1317. Reduce the memory footprint of Rumen objects by interning
  4806. host Strings. (Hong Tang via cdouglas)
  4807. MAPREDUCE-1097. Add support for Vertica 3.5 to its contrib module. (Omer
  4808. Trajman via cdouglas)
  4809. MAPREDUCE-1627. HadoopArchives should not uses a method in DistCp.
  4810. (szetszwo)
  4811. MAPREDUCE-1198. Alternatively schedule different types of tasks in
  4812. fair share scheduler. (Scott Chen via matei)
  4813. MAPREDUCE-707. Provide a jobconf property for explicitly assigning a job to
  4814. a pool in the Fair Scheduler. (Alan Heirich via matei)
  4815. MAPREDUCE-947. Added commitJob and abortJob apis to OutputCommitter.
  4816. Enhanced FileOutputCommitter to create a _SUCCESS file for successful
  4817. jobs. (Amar Kamat & Jothi Padmanabhan via acmurthy)
  4818. MAPREDUCE-1103. Added more metrics to Jobtracker. (sharad)
  4819. MAPREDUCE-1048. Add occupied/reserved slot usage summary on jobtracker UI.
  4820. (Amareshwari Sriramadasu and Hemanth Yamijala via sharad)
  4821. MAPREDUCE-1090. Modified log statement in TaskMemoryManagerThread to
  4822. include task attempt id. (yhemanth)
  4823. MAPREDUCE-1189. Reduce ivy console output to ovservable level (cos)
  4824. MAPREDUCE-1167. ProcfsBasedProcessTree collects rss memory information.
  4825. (Scott Chen via dhruba)
  4826. MAPREDUCE-1231. Added a new DistCp option, -skipcrccheck, so that the CRC
  4827. check during setup can be skipped. (Jothi Padmanabhan via szetszwo)
  4828. MAPREDUCE-1190. Add package documentation for BBP example.
  4829. (Tsz Wo (Nicholas) Sze via cdouglas)
  4830. MAPREDUCE-1119. When tasks fail to report status, show tasks's stack dump
  4831. before killing. (Aaron Kimball via tomwhite)
  4832. MAPREDUCE-1185. Redirect running job url to history url if job is already
  4833. retired. (Amareshwari Sriramadasu and Sharad Agarwal via sharad)
  4834. MAPREDUCE-1050. Introduce a mock object testing framework. (tomwhite)
  4835. MAPREDUCE-1084. Implementing aspects development and fault injeciton
  4836. framework for MapReduce. (Sreekanth Ramakrishnan via cos)
  4837. MAPREDUCE-1209. Move common specific part of the test TestReflectionUtils
  4838. out of mapred into common. (Todd Lipcon via tomwhite)
  4839. MAPREDUCE-967. TaskTracker does not need to fully unjar job jars.
  4840. (Todd Lipcon via tomwhite)
  4841. MAPREDUCE-1083. Changes in MapReduce so that group information of users
  4842. can be refreshed in the JobTracker via command line.
  4843. (Boris Shkolnik via ddas)
  4844. MAPREDUCE-181. Changes the job submission process to be secure.
  4845. (Devaraj Das)
  4846. MAPREDUCE-1250. Refactors the JobToken to use Common's Token interface.
  4847. (Kan Zhang via ddas)
  4848. MAPREDUCE-896. Enhance tasktracker to cleanup files that might have
  4849. been created by user tasks with non-writable permissions.
  4850. (Ravi Gummadi via yhemanth)
  4851. MAPREDUCE-372. Change org.apache.hadoop.mapred.lib.ChainMapper/Reducer
  4852. to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
  4853. MAPREDUCE-1295. Add a tool in Rumen for folding and manipulating job
  4854. traces. (Dick King via cdouglas)
  4855. MAPREDUCE-1302. TrackerDistributedCacheManager deletes file
  4856. asynchronously, thus reducing task initialization delays.
  4857. (Zheng Shao via dhruba)
  4858. MAPREDUCE-1218. TaskTrackers send cpu and memory usage of
  4859. node to JobTracker. (Scott Chen via dhruba)
  4860. MAPREDUCE-847. Fix Releaseaudit warning count to zero
  4861. (Giridharan Kesavan)
  4862. MAPREDUCE-1337. Use generics in StreamJob to improve readability of that
  4863. class. (Kay Kay via cdouglas)
  4864. MAPREDUCE-361. Port terasort example to the new mapreduce API. (Amareshwari
  4865. Sriramadasu via cdouglas)
  4866. MAPREDUCE-1367. LocalJobRunner should support parallel mapper execution.
  4867. (Aaron Kimball via tomwhite)
  4868. MAPREDUCE-64. Eliminate io.sort.record.percent from MapTask configuration.
  4869. (cdouglas)
  4870. MAPREDUCE-1440. Replace the long user name in MapReduce with the local
  4871. name. (omalley)
  4872. MAPREDUCE-1470. Move delegation tokens from HDFS to Common so that
  4873. MapReduce can use them too. (omalley)
  4874. MAPREDUCE-1425. Reduce memory usage by archive. (mahadev via szetszwo)
  4875. MAPREDUCE-1441. Trim whitespace from directory lists pulled from the
  4876. configuration. (Todd Lipcon via cdouglas)
  4877. MAPREDUCE-1309. Refactor Rumen trace generator to improve code structure
  4878. and add extensible support for log formats. (Dick King via cdouglas)
  4879. MAPREDUCE-1503. Delegation token renewing and cancelling should provide
  4880. meaningful exceptions when there are failures instead of returning
  4881. false. (omalley)
  4882. HADOOP-6579. Upgrade commons-codec library to 1.4. (omalley)
  4883. MAPREDUCE-1423. Improve performance of CombineFileInputFormat when multiple
  4884. pools are configured. (Dhruba Borthakur via zshao)
  4885. MAPREDUCE-1454. Quote user supplied strings in Tracker servlets. (cdouglas)
  4886. MAPREDUCE-1408. Add customizable job submission policies to Gridmix. (Rahul
  4887. Singh via cdouglas)
  4888. MAPREDUCE-1527. Better warning logged when mapred.queue.names is
  4889. overshadowed by mapred-queues.xml. (Hong Tang via acmurthy)
  4890. MAPREDUCE-1403. Save the size and number of distributed cache artifacts in
  4891. the configuration. (Arun Murthy via cdouglas)
  4892. MAPREDUCE-1482. Truncate state string and diagnostic information in
  4893. TaskStatus. (Amar Kamat via szetszwo)
  4894. MAPREDUCE-1593. [Rumen] Improvements to random seed generation (tamas via
  4895. mahadev)
  4896. MAPREDUCE-1460. Oracle support in DataDrivenDBInputFormat.
  4897. (Aaron Kimball via tomwhite)
  4898. MAPREDUCE-1569. Pass configuration through mocked contexts in MRUnit.
  4899. (Chris White via cdouglas)
  4900. MAPREDUCE-1590. Move HarFileSystem from Hadoop Common to Mapreduce tools.
  4901. (mahadev)
  4902. MAPREDUCE-1629. Get rid of fakeBlockLocations() on HarFileSystem, since
  4903. it's not used (mahadev)
  4904. MAPREDUCE-1489. DataDrivenDBInputFormat should not query the database
  4905. when generating only one split. (Aaron Kimball via tomwhite)
  4906. MAPREDUCE-1514. Add documentation on replication, permissions, new options,
  4907. limitations and internals of har. (mahadev via szetszwo)
  4908. MAPREDUCE-1428. Make block size and the size of archive created files
  4909. configurable. (mahadev via szetszwo)
  4910. MAPREDUCE-1656. JobStory should provide queue info. (hong via mahadev)
  4911. MAPREDUCE-1466. Record number of files processed in FileInputFormat in the
  4912. Configuration for offline analysis. (Luke Lu and Arun Murthy via cdouglas)
  4913. MAPREDUCE-1538. TrackerDistributedCacheManager manages the
  4914. number of files. (Scott Chen via dhruba)
  4915. MAPREDUCE-1673. Scripts to start and stop RaidNode.
  4916. (Rodrigo Schmidt via dhruba)
  4917. MAPREDUCE-1659. RaidNode writes temp files on configured tmp directory and
  4918. add random numbers to their names to avoid conflicts
  4919. (Rodrigo Schmidt via dhruba)
  4920. MAPREDUCE-1221. Allow admins to control physical memory limits per-task
  4921. and per-node. (Scott Chen via acmurthy)
  4922. MAPREDUCE-1065. Update mapred tutorial to use the new API. (Aaron Kimball
  4923. via cdouglas)
  4924. MAPREDUCE-1304. Add a task counter tracking time spent in GC. (Aaron
  4925. Kimball via cdouglas)
  4926. MAPREDUCE-1570. Add grouping comparators to MRUnit. (Chris White via
  4927. cdouglas)
  4928. MAPREDUCE-1650. Exclude Private elements from generated MapReduce
  4929. Javadoc. (tomwhite)
  4930. MAPREDUCE-1625. Improve grouping of packages in Javadoc. (tomwhite)
  4931. MAPREDUCE-1417. Forrest documentation should be updated to reflect
  4932. the changes in MAPREDUCE-744. (Ravi Gummadi via vinodkv)
  4933. MAPREDUCE-1568. TrackerDistributedCacheManager should clean up cache
  4934. in a background thread. (Scott Chen via zshao)
  4935. MAPREDUCE-1749. Move configuration strings out of JobContext so that it
  4936. can be made public stable. (omalley)
  4937. MAPREDUCE-1623. Apply audience and stability notations to Hadoop
  4938. Map-Reduce. (tomwhite via acmurthy)
  4939. MAPREDUCE-1751. Change MapReduce to depend on Hadoop 'common' artifacts
  4940. instead of 'core'. (tomwhite)
  4941. MAPREDUCE-1535. Replace usage of FileStatus#isDir(). (Eli Collins via
  4942. tomwhite)
  4943. MAPREDUCE-1832. Allow file sizes less than 1MB in DFSIO benchmark. (shv)
  4944. MAPREDUCE-1404. Move Cluster-Setup and Single-Node-Setup Docs from
  4945. MapReduce to Common. (tomwhite)
  4946. MAPREDUCE-1697. Document the behavior of -file option and deprecate it
  4947. in favour of -files option in streaming. (Amareshwari Sriramadasu
  4948. via vinodkv)
  4949. MAPREDUCE-1033. Resolve location of scripts and configuration files after
  4950. project split. (tomwhite)
  4951. MAPREDUCE-1018. Document changes to the memory management and scheduling
  4952. model. (Hemanth Yamijala via vinodkv)
  4953. MAPREDUCE-1896. [Herriot] New property for multi user list. (Vinay Thota
  4954. via cos)
  4955. MAPREDUCE-1812. New properties for suspend and resume process. (Vinay
  4956. Thota via cos)
  4957. OPTIMIZATIONS
  4958. MAPREDUCE-270. Fix the tasktracker to optionally send an out-of-band
  4959. heartbeat on task-completion for better job-latency. (acmurthy)
  4960. Configuration changes:
  4961. add mapreduce.tasktracker.outofband.heartbeat
  4962. MAPREDUCE-1186. Modified code in distributed cache to set permissions
  4963. only on required set of localized paths.
  4964. (Amareshwari Sriramadasu via yhemanth)
  4965. MAPREDUCE-1501. FileInputFormat supports multi-level, recursive
  4966. directory listing. (Zheng Shao via dhruba)
  4967. MAPREDUCE-1556. upgrade to Avro 1.3.0. (cutting via tomwhite)
  4968. MAPREDUCE-1613. Install/deploy source jars to Maven repo
  4969. (Patrick Angeles via ddas)
  4970. MAPREDUCE-1610. Forrest documentation should be updated to reflect
  4971. the changes in MAPREDUCE-856. (Ravi Gummadi via vinodkv)
  4972. MAPREDUCE-1853. Adds caching for TaskAttemptContext in MultipleOutputs.
  4973. (Torsten Curdt via amareshwari)
  4974. BUG FIXES
  4975. MAPREDUCE-878. Rename fair scheduler design doc to
  4976. fair-scheduler-design-doc.tex and add Apache license header (matei)
  4977. HADOOP-4687. MapReduce is split from Hadoop Core. It is a subproject under
  4978. Hadoop (Owen O'Malley)
  4979. HADOOP-6096. Fix Eclipse project and classpath files following project
  4980. split. (tomwhite)
  4981. MAPREDUCE-419. Reconcile mapred.userlog.limit.kb defaults in configuration
  4982. and code. (Philip Zeyliger via cdouglas)
  4983. MAPREDUCE-2. Fixes a bug in KeyFieldBasedPartitioner in handling empty
  4984. keys. (Amar Kamat via sharad)
  4985. MAPREDUCE-130. Delete the jobconf copy from the log directory of the
  4986. JobTracker when the job is retired. (Amar Kamat via sharad)
  4987. MAPREDUCE-657. Fix hardcoded filesystem problem in CompletedJobStatusStore.
  4988. (Amar Kamat via sharad)
  4989. MAPREDUCE-179. Update progress in new RecordReaders. (cdouglas)
  4990. MAPREDUCE-658. Replace NPE in distcp with a meaningful error message when
  4991. the source path does not exist. (Ravi Gummadi via cdouglas)
  4992. MAPREDUCE-671. Update ignore list to include untracked, generated
  4993. build artifacts and config files. (cdouglas)
  4994. MAPREDUCE-433. Use more reliable counters in TestReduceFetch. (cdouglas)
  4995. MAPREDUCE-124. Fix a bug in failure handling of abort task of
  4996. OutputCommiter. (Amareshwari Sriramadasu via sharad)
  4997. MAPREDUCE-694. Fix to add jsp-api jars to capacity-scheduler classpath.
  4998. (Giridharan Kesavan)
  4999. MAPREDUCE-702. Fix eclipse-plugin jar target (Giridharan Kesavan)
  5000. MAPREDUCE-522. Replace TestQueueCapacities with simpler test case to
  5001. test integration between capacity scheduler and MR framework.
  5002. (Sreekanth Ramakrishnan via yhemanth)
  5003. MAPREDUCE-683. Fixes an initialization problem in the JobHistory.
  5004. The initialization of JobHistoryFilesManager is now done in the
  5005. JobHistory.init call. (Amar Kamat via ddas)
  5006. MAPREDUCE-708. Fixes a bug to allow updating the reason for
  5007. blacklisting a node on the JobTracker UI.
  5008. (Sreekanth Ramakrishnan via yhemanth)
  5009. MAPREDUCE-709. Fixes message displayed for a blacklisted node where
  5010. the reason for blacklisting is due to the health check script
  5011. timing out. (Sreekanth Ramakrishnan via yhemanth)
  5012. MAPREDUCE-676. Existing diagnostic rules fail for MAP ONLY jobs.
  5013. (Suhas Gogate via tomwhite)
  5014. MAPREDUCE-722. Fixes a bug with tasktracker reservations for
  5015. high memory jobs in capacity scheduler.
  5016. (Vinod Kumar Vavilapalli via yhemanth)
  5017. HADOOP-6090. Updates gridmix script to use new mapreduce api output
  5018. format. (Amareshwari Sriramadasu via sharad)
  5019. MAPREDUCE-732. Removed spurious log statements in the node
  5020. blacklisting logic. (Sreekanth Ramakrishnan via yhemanth)
  5021. MAPREDUCE-734. Fix a ConcurrentModificationException in unreserving
  5022. unused reservations for a job when it completes.
  5023. (Arun Murthy and Sreekanth Ramakrishnan via yhemanth)
  5024. MAPREDUCE-733. Fix a RuntimeException while unreserving trackers
  5025. that are blacklisted for a job.
  5026. (Arun Murthy and Sreekanth Ramakrishnan via yhemanth)
  5027. MAPREDUCE-677. Fix timeout in TestNodeRefresh. (Amar Kamat via
  5028. sharad)
  5029. MAPREDUCE-153. Fix timeout in TestJobInProgressListener. (Amar
  5030. Kamat via sharad)
  5031. MAPREDUCE-742. Fix output messages and java comments in the Pi related
  5032. examples. (szetszwo)
  5033. MAPREDUCE-565. Fix partitioner to work with new API. (Owen O'Malley via
  5034. cdouglas)
  5035. MAPREDUCE-680. Fix so MRUnit can handle reuse of Writable objects.
  5036. (Aaron Kimball via johan)
  5037. MAPREDUCE-18. Puts some checks for cross checking whether a reduce
  5038. task gets the correct shuffle data. (Ravi Gummadi via ddas)
  5039. MAPREDUCE-771. Fix scheduling of setup and cleanup tasks to use
  5040. free slots instead of tasks for scheduling. (yhemanth)
  5041. MAPREDUCE-717. Fixes some corner case issues in speculative
  5042. execution heuristics. (Devaraj Das)
  5043. MAPREDUCE-716. Make DBInputFormat work with Oracle. (Aaron Kimball
  5044. via tomwhite)
  5045. MAPREDUCE-735. Fixes a problem in the KeyFieldHelper to do with
  5046. the end index for some inputs (Amar Kamat via ddas)
  5047. MAPREDUCE-682. Removes reservations on tasktrackers which are
  5048. blacklisted. (Sreekanth Ramakrishnan via yhemanth)
  5049. MAPREDUCE-743. Fixes a problem to do with progress reporting
  5050. in the map phase. (Ravi Gummadi via ddas)
  5051. MAPREDUCE-765. Eliminate the deprecated warnings introduced by H-5438.
  5052. (He Yongqiang via szetszwo)
  5053. MAPREDUCE-383. Fix a bug in Pipes combiner due to bytes count not
  5054. getting reset after the spill. (Christian Kunz via sharad)
  5055. MAPREDUCE-809. Fix job-summary logs to correctly record status of FAILED
  5056. and KILLED jobs. (acmurthy)
  5057. MAPREDUCE-792. Fix unchecked warnings in DBInputFormat. (Aaron Kimball
  5058. via szetszwo)
  5059. MAPREDUCE-760. Fix a timing issue in TestNodeRefresh. (Amar Kamat via
  5060. sharad)
  5061. MAPREDUCE-40. Keep memory management backwards compatible for job
  5062. configuration parameters and limits. (Rahul Kumar Singh via yhemanth)
  5063. MAPREDUCE-587. Fixes a OOM issue in TestStreamingExitStatus.
  5064. (Amar Kamat via ddas)
  5065. MAPREDUCE-408. Fixes an assertion problem in TestKillSubProcesses
  5066. (Ravi Gummadi via ddas)
  5067. MAPREDUCE-659. Fix gridmix2 compilation. (Giridharan Kesavan)
  5068. MAPREDUCE-796. Fixes a ClassCastException in an exception log in
  5069. MultiThreadedMapRunner. (Amar Kamat via ddas)
  5070. MAPREDUCE-808. Fixes a serialization problem in TypedBytes.
  5071. (Klaas Bosteels via ddas)
  5072. MAPREDUCE-845. Fix a findbugs heap size problem in build.xml and add
  5073. a new property findbugs.heap.size. (Lee Tucker via szetszwo)
  5074. MAPREDUCE-838. Fixes a problem in the way commit of task outputs
  5075. happens. The bug was that even if commit failed, the task would
  5076. be declared as successful. (Amareshwari Sriramadasu via ddas)
  5077. MAPREDUCE-813. Updates Streaming and M/R tutorial documents.
  5078. (Corinne Chandel via ddas)
  5079. MAPREDUCE-805. Fixes some deadlocks in the JobTracker due to the fact
  5080. the JobTracker lock hierarchy wasn't maintained in some JobInProgress
  5081. method calls. (Amar Kamat via ddas)
  5082. MAPREDUCE-799. Fixes so all of the MRUnit self-tests run.
  5083. (Aaron Kimball via johan)
  5084. MAPREDUCE-848. Fixes a problem to do with TestCapacityScheduler
  5085. failing (Amar Kamat via ddas)
  5086. MAPREDUCE-840. DBInputFormat leaves open transaction.
  5087. (Aaron Kimball via tomwhite)
  5088. MAPREDUCE-859. Adds Avro and its dependencies required by Hadoop
  5089. common. (Ravi Gummadi via sharad)
  5090. MAPREDUCE-867. Fix ivy conf to look for avro jar from maven repo.
  5091. (Giridharan Kesavan)
  5092. MAPREDUCE-877. Added avro as a dependency to contrib ivy settings.
  5093. (Tsz Wo (Nicholas) Sze via yhemanth)
  5094. MAPREDUCE-852. In build.xml, remove the Main-Class, which is incorrectly
  5095. set in tools, and rename the target "tools-jar" to "tools". (szetszwo)
  5096. MAPREDUCE-773. Sends progress reports for compressed gzip inputs in maps.
  5097. Fixes a native direct buffer leak in LineRecordReader classes.
  5098. (Hong Tang and ddas)
  5099. MAPREDUCE-832. Reduce number of warning messages printed when
  5100. deprecated memory variables are used. (Rahul Kumar Singh via yhemanth)
  5101. MAPREDUCE-745. Fixes a testcase problem to do with generation of JobTracker
  5102. IDs. (Amar Kamat via ddas)
  5103. MAPREDUCE-834. Enables memory management on tasktrackers when old
  5104. memory management parameters are used in configuration.
  5105. (Sreekanth Ramakrishnan via yhemanth)
  5106. MAPREDUCE-818. Fixes Counters#getGroup API. (Amareshwari Sriramadasu
  5107. via sharad)
  5108. MAPREDUCE-807. Handles the AccessControlException during the deletion of
  5109. mapred.system.dir in the JobTracker. The JobTracker will bail out if it
  5110. encounters such an exception. (Amar Kamat via ddas)
  5111. MAPREDUCE-430. Fix a bug related to task getting stuck in case of
  5112. OOM error. (Amar Kamat via ddas)
  5113. MAPREDUCE-871. Fix ownership of Job/Task local files to have correct
  5114. group ownership according to the egid of the tasktracker.
  5115. (Vinod Kumar Vavilapalli via yhemanth)
  5116. MAPREDUCE-911. Fix a bug in TestTaskFail related to speculative
  5117. execution. (Amareshwari Sriramadasu via sharad)
  5118. MAPREDUCE-687. Fix an assertion in TestMiniMRMapRedDebugScript.
  5119. (Amareshwari Sriramadasu via sharad)
  5120. MAPREDUCE-924. Fixes the TestPipes testcase to use Tool.
  5121. (Amareshwari Sriramadasu via sharad)
  5122. MAPREDUCE-903. Add Avro jar to eclipse classpath.
  5123. (Philip Zeyliger via tomwhite)
  5124. MAPREDUCE-943. Removes a testcase in TestNodeRefresh that doesn't make
  5125. sense in the new Job recovery model. (Amar Kamat via ddas)
  5126. MAPREDUCE-764. TypedBytesInput's readRaw() does not preserve custom type
  5127. codes. (Klaas Bosteels via tomwhite)
  5128. HADOOP-6243. Fixes a NullPointerException in handling deprecated keys.
  5129. (Sreekanth Ramakrishnan via yhemanth)
  5130. MAPREDUCE-968. NPE in distcp encountered when placing _logs directory on
  5131. S3FileSystem. (Aaron Kimball via tomwhite)
  5132. MAPREDUCE-826. harchive doesn't use ToolRunner / harchive returns 0 even
  5133. if the job fails with exception (koji Noguchi via mahadev)
  5134. MAPREDUCE-839. unit test TestMiniMRChildTask fails on mac os-x (hong tang
  5135. via mahadev)
  5136. MAPREDUCE-112. Add counters for reduce input, output records to the new API.
  5137. (Jothi Padmanabhan via cdouglas)
  5138. MAPREDUCE-648. Fix two distcp bugs: (1) it should not launch a job if all
  5139. src paths are directories, and (2) it does not skip copying when updating
  5140. a single file. (Ravi Gummadi via szetszwo)
  5141. MAPREDUCE-946. Fix a regression in LineRecordReader where the
  5142. maxBytesToConsume parameter is not set correctly. (cdouglas)
  5143. MAPREDUCE-977. Missing jackson jars from Eclipse template. (tomwhite)
  5144. MAPREDUCE-988. Fix a packaging issue in the contrib modules. (Hong Tang via
  5145. cdouglas)
  5146. MAPREDUCE-971. distcp does not always remove distcp.tmp.dir. (Aaron Kimball
  5147. via tomwhite)
  5148. MAPREDUCE-995. Fix a bug in JobHistory where tasks completing after the job
  5149. is closed cause a NPE. (Jothi Padmanabhan via cdouglas)
  5150. MAPREDUCE-953. Fix QueueManager to dump queue configuration in JSON format.
  5151. (V.V. Chaitanya Krishna via yhemanth)
  5152. MAPREDUCE-645. Prevent distcp from running a job when the destination is a
  5153. file, but the source is not. (Ravi Gummadi via cdouglas)
  5154. MAPREDUCE-1002. Flushed writer in JobQueueClient so queue information is
  5155. printed correctly. (V.V. Chaitanya Krishna via yhemanth)
  5156. MAPREDUCE-1003. Fix compilation problem in eclipse plugin when
  5157. eclipse.home is set. (Ravi Gummadi via yhemanth)
  5158. MAPREDUCE-941. Vaidya script fails on Solaris. (Chad Metcalf
  5159. via tomwhite)
  5160. MAPREDUCE-912. Add and standardize Apache license headers. (Chad Metcalf
  5161. via cdouglas)
  5162. MAPREDUCE-1022. Fix compilation of vertica testcases. (Vinod Kumar
  5163. Vavilapalli via acmurthy)
  5164. MAPREDUCE-1000. Handle corrupt history files in JobHistory.initDone().
  5165. (Jothi Padmanabhan via sharad)
  5166. MAPREDUCE-1028. Fixed number of slots occupied by cleanup tasks to one
  5167. irrespective of slot size for the job.
  5168. (Ravi Gummadi via yhemanth)
  5169. MAPREDUCE-964. Fixed start and finish times of TaskStatus to be
  5170. consistent, thereby fixing inconsistencies in metering tasks.
  5171. (Sreekanth Ramakrishnan via yhemanth)
  5172. MAPREDUCE-1076. Deprecate ClusterStatus and add javadoc in ClusterMetrics.
  5173. (Amareshwari Sriramadasu via sharad)
  5174. MAPREDUCE-979. Fixed JobConf APIs related to memory parameters to return
  5175. values of new configuration variables when deprecated variables are
  5176. disabled. (Sreekanth Ramakrishnan via yhemanth)
  5177. MAPREDUCE-1030. Modified scheduling algorithm to return a map and reduce
  5178. task per heartbeat in the capacity scheduler.
  5179. (Rahul Kumar Singh via yhemanth)
  5180. MAPREDUCE-1071. Use DataInputStream rather than FSDataInputStream in the
  5181. JobHistory EventReader. (Hong Tang via cdouglas)
  5182. MAPREDUCE-986. Fix Rumen to work with truncated task lines. (Dick King via
  5183. cdouglas)
  5184. MAPREDUCE-1029. Fix failing TestCopyFiles by restoring the unzipping of
  5185. HDFS webapps from the hdfs jar. (Aaron Kimball and Jothi Padmanabhan via
  5186. cdouglas)
  5187. MAPREDUCE-769. Make findbugs and javac warnings to zero.
  5188. (Amareshwari Sriramadasu via sharad)
  5189. MAPREDUCE-1104. Initialize RecoveryManager in JobTracker cstr called by
  5190. Mumak. (Hong Tang via cdouglas)
  5191. MAPREDUCE-1061. Add unit test validating byte specifications for gridmix
  5192. jobs. (cdouglas)
  5193. MAPREDUCE-1077. Fix Rumen so that truncated tasks do not mark the job as
  5194. successful. (Dick King via cdouglas)
  5195. MAPREDUCE-1041. Make TaskInProgress::taskStatuses map package-private.
  5196. (Jothi Padmanabhan via cdouglas)
  5197. MAPREDUCE-1070. Prevent a deadlock in the fair scheduler servlet.
  5198. (Todd Lipcon via cdouglas)
  5199. MAPREDUCE-1086. Setup Hadoop logging environment for tasks to point to
  5200. task related parameters. (Ravi Gummadi via yhemanth)
  5201. MAPREDUCE-1105. Remove max limit configuration in capacity scheduler in
  5202. favor of max capacity percentage thus allowing the limit to go over
  5203. queue capacity. (Rahul Kumar Singh via yhemanth)
  5204. MAPREDUCE-1016. Make the job history log format JSON. (cutting)
  5205. MAPREDUCE-1038. Weave Mumak aspects only if related files have changed.
  5206. (Aaron Kimball via cdouglas)
  5207. MAPREDUCE-1163. Remove unused, hard-coded paths from libhdfs. (Allen
  5208. Wittenauer via cdouglas)
  5209. MAPREDUCE-962. Fix a NullPointerException while killing task process
  5210. trees. (Ravi Gummadi via yhemanth)
  5211. MAPREDUCE-1177. Correct setup/cleanup inversion in
  5212. JobTracker::getTaskReports. (Vinod Kumar Vavilapalli via cdouglas)
  5213. MAPREDUCE-1178. Fix ClassCastException in MultipleInputs by adding
  5214. a DelegatingRecordReader. (Amareshwari Sriramadasu and Jay Booth
  5215. via sharad)
  5216. MAPREDUCE-1068. Fix streaming job to show proper message if file is
  5217. is not present. (Amareshwari Sriramadasu via sharad)
  5218. MAPREDUCE-1147. Add map output counters to new API. (Amar Kamat via
  5219. cdouglas)
  5220. MAPREDUCE-915. The debug scripts are run as the job user. (ddas)
  5221. MAPREDUCE-1007. Fix NPE in CapacityTaskScheduler.getJobs().
  5222. (V.V.Chaitanya Krishna via sharad)
  5223. MAPREDUCE-28. Refactor TestQueueManager and fix default ACLs.
  5224. (V.V.Chaitanya Krishna and Rahul K Singh via sharad)
  5225. MAPREDUCE-1182. Fix overflow in reduce causing allocations to exceed the
  5226. configured threshold. (cdouglas)
  5227. MAPREDUCE-1239. Fix contrib components build dependencies.
  5228. (Giridharan Kesavan and omalley)
  5229. MAPREDUCE-787. Fix JobSubmitter to honor user given symlink path.
  5230. (Amareshwari Sriramadasu via sharad)
  5231. MAPREDUCE-1249. Update config default value for socket read timeout to
  5232. match code default. (Amareshwari Sriramadasu via cdouglas)
  5233. MAPREDUCE-1161. Remove ineffective synchronization in NotificationTestCase.
  5234. (Owen O'Malley via cdouglas)
  5235. MAPREDUCE-1244. Fix eclipse-plugin's build dependencies. (gkesavan)
  5236. MAPREDUCE-1075. Fix JobTracker to not throw an NPE for a non-existent
  5237. queue. (V.V.Chaitanya Krishna via yhemanth)
  5238. MAPREDUCE-754. Fix NPE in expiry thread when a TT is lost. (Amar Kamat
  5239. via sharad)
  5240. MAPREDUCE-1074. Document Reducer mark/reset functionality. (Jothi
  5241. Padmanabhan via cdouglas)
  5242. MAPREDUCE-1267. Fix typo in mapred-default.xml. (Todd Lipcon via cdouglas)
  5243. MAPREDUCE-952. Remove inadvertently reintroduced Task.Counter enum. (Jothi
  5244. Padmanabhan via cdouglas)
  5245. MAPREDUCE-1230. Fix handling of null records in VerticaInputFormat. (Omer
  5246. Trajman via cdouglas)
  5247. MAPREDUCE-1171. Allow shuffle retries and read-error reporting to be
  5248. configurable. (Amareshwari Sriramadasu via acmurthy)
  5249. MAPREDUCE-879. Fix broken unit test TestTaskTrackerLocalization on MacOS.
  5250. (Sreekanth Ramakrishnan via yhemanth)
  5251. MAPREDUCE-1124. Fix imprecise byte counts in Gridmix. (cdouglas)
  5252. MAPREDUCE-1222. Add an option to exclude numeric IP addresses in topologies
  5253. processed by Mumak. (Hong Tang via cdouglas)
  5254. MAPREDUCE-1284. Fix fts_open() call in task-controller that was failing
  5255. LinuxTaskController unit tests. (Ravi Gummadi via yhemanth)
  5256. MAPREDUCE-1143. Fix running task counters to be updated correctly
  5257. when speculative attempts are running for a TIP.
  5258. (Rahul Kumar Singh via yhemanth)
  5259. MAPREDUCE-1241. Use a default queue configuration in JobTracker when
  5260. mapred-queues.xml is unavailable. (Todd Lipcon via cdouglas)
  5261. MAPREDUCE-1301. Fix set up of permission checking script used in
  5262. localization tests. (Amareshwari Sriramadasu via yhemanth)
  5263. MAPREDUCE-1286. Remove quoting from client opts in TaskRunner. (Yuri
  5264. Pradkin via cdouglas)
  5265. MAPREDUCE-1059. Use distcp.bytes.per.map when adding sync markers in
  5266. distcp. (Aaron Kimball via cdouglas)
  5267. MAPREDUCE-1009. Update forrest documentation describing hierarchical
  5268. queues. (Vinod Kumar Vavilapalli via yhemanth)
  5269. MAPREDUCE-1342. Fixed deadlock in global blacklisting of tasktrackers.
  5270. (Amareshwari Sriramadasu via acmurthy)
  5271. MAPREDUCE-1316. Fixes a memory leak of TaskInProgress instances in
  5272. the jobtracker. (Amar Kamat via yhemanth)
  5273. MAPREDUCE-1359. TypedBytes TestIO doesn't mkdir its test dir first.
  5274. (Anatoli Fomenko via cos)
  5275. MAPREDUCE-1314. Correct errant mapreduce.x.mapreduce.x replacements from
  5276. bulk change. (Amareshwari Sriramadasu via cdouglas)
  5277. MAPREDUCE-1365. Restore accidentally renamed test in
  5278. TestTaskTrackerBloacklisting. (Amareshwari Sriramadasu via cdouglas)
  5279. MAPREDUCE-1406. Fix spelling of JobContext.MAP_COMBINE_MIN_SPILLS.
  5280. (cdouglas)
  5281. MAPREDUCE-1369. JUnit tests should never depend on anything in conf
  5282. (Anatoli Fomenko via cos)
  5283. MAPREDUCE-1412. Fix timer granularity issue causing failures in
  5284. TestTaskTrackerBlacklisting. (cdouglas)
  5285. MAPREDUCE-1448. Respect --config option in Mumak script. (Hong Tang via
  5286. cdouglas)
  5287. MAPREDUCE-1251. c++ utils doesn't compile. (Eli Collins via tomwhite)
  5288. MAPREDUCE-1522. FileInputFormat may use the default FileSystem for the
  5289. input path. (Tsz Wo (Nicholas), SZE via cdouglas)
  5290. MAPREDUCE-1407. Update javadoc in mapreduce.{Mapper,Reducer} to match
  5291. actual usage. (Benoit Sigoure via cdouglas)
  5292. MAPREDUCE-1258. Fix fair scheduler event log not logging job info.
  5293. (matei)
  5294. MAPREDUCE-1089. Fix NPE in fair scheduler preemption when tasks are
  5295. scheduled but not running. (Todd Lipcon via matei)
  5296. MAPREDUCE-1014. Fix the libraries for common and hdfs. (omalley)
  5297. MAPREDUCE-1111. JT Jetty UI not working if we run mumak.sh
  5298. off packaged distribution directory. (hong tang via mahadev)
  5299. MAPREDUCE-1133. Eclipse .classpath template has outdated jar files and is
  5300. missing some new ones. (cos)
  5301. MAPREDUCE-1098. Fixed the distributed-cache to not do i/o while holding a
  5302. global lock. (Amareshwari Sriramadasu via acmurthy)
  5303. MAPREDUCE-1158. Fix JT running maps and running reduces metrics.
  5304. (sharad)
  5305. MAPREDUCE-1160. Reduce verbosity of log lines in some Map/Reduce classes
  5306. to avoid filling up jobtracker logs on a busy cluster.
  5307. (Ravi Gummadi and Hong Tang via yhemanth)
  5308. MAPREDUCE-1153. Fix tasktracker metrics when trackers are decommissioned.
  5309. (sharad)
  5310. MAPREDUCE-1128. Fix MRUnit to prohibit iterating over values twice. (Aaron
  5311. Kimball via cdouglas)
  5312. MAPREDUCE-665. Move libhdfs to HDFS subproject. (Eli Collins via dhruba)
  5313. MAPREDUCE-1196. Fix FileOutputCommitter to use the deprecated cleanupJob
  5314. api correctly. (acmurthy)
  5315. MAPREDUCE-1244. Fix eclipse-plugin's build dependencies. (gkesavan)
  5316. MAPREDUCE-1140. Fix DistributedCache to not decrement reference counts for
  5317. unreferenced files in error conditions.
  5318. (Amareshwari Sriramadasu via yhemanth)
  5319. MAPREDUCE-1245. Fix TestFairScheduler failures by instantiating lightweight
  5320. Jobtracker. (sharad)
  5321. MAPREDUCE-1260. Update Eclipse configuration to match changes to Ivy
  5322. configuration. (Edwin Chan via cos)
  5323. MAPREDUCE-1152. Distinguish between failed and killed tasks in
  5324. JobTrackerInstrumentation. (Sharad Agarwal via cdouglas)
  5325. MAPREDUCE-1285. In DistCp.deleteNonexisting(..), get class from the
  5326. parameter instead of using FileStatus.class. (Peter Romianowski via
  5327. szetszwo)
  5328. MAPREDUCE-1294. Build fails to pull latest hadoop-core-* artifacts (cos)
  5329. MAPREDUCE-1213. TaskTrackers restart is faster because it deletes
  5330. distributed cache directory asynchronously. (Zheng Shao via dhruba)
  5331. MAPREDUCE-1265. The task attempt error log prints the name of the
  5332. tasktracker machine. (Scott Chen via dhruba)
  5333. MAPREDUCE-1201. ProcfsBasedProcessTree collects CPU usage information.
  5334. (Scott Chen via dhruba)
  5335. MAPREDUCE-1326. fi tests don't use fi-site.xml (cos)
  5336. MAPREDUCE-1165. Replace non-portable function name with C99 equivalent.
  5337. (Allen Wittenauer via cdouglas)
  5338. MAPREDUCE-1331. Fixes a typo in a testcase (Devaraj Das)
  5339. MAPREDUCE-1293. AutoInputFormat doesn't work with non-default FileSystems.
  5340. (Andrew Hitchcock via tomwhite)
  5341. MAPREDUCE-1131. Using profilers other than hprof can cause JobClient to
  5342. report job failure. (Aaron Kimball via tomwhite)
  5343. MAPREDUCE-1155. Streaming tests swallow exceptions.
  5344. (Todd Lipcon via tomwhite)
  5345. MAPREDUCE-1212. Mapreduce contrib project ivy dependencies are not included
  5346. in binary target. (Aaron Kimball via tomwhite)
  5347. MAPREDUCE-1388. Move the HDFS RAID package from HDFS to MAPREDUCE.
  5348. (Eli Collins via dhruba)
  5349. MAPREDUCE-1322. Defines default value for staging directory to be user
  5350. based fixing a failing streaming test.
  5351. (Devaraj Das and Amareshwari Sriramadasu via yhemanth)
  5352. MAPREDUCE-899. Modified LinuxTaskController to check that task-controller
  5353. has right permissions and ownership before performing any actions.
  5354. (Amareshwari Sriramadasu via yhemanth)
  5355. MAPREDUCE-1443. DBInputFormat can leak connections.
  5356. (Aaron Kimball via tomwhite)
  5357. MAPREDUCE-1457. Fixes JobTracker to get the FileSystem object within
  5358. getStagingAreaDir within a privileged block. Fixes Child.java to use the
  5359. appropriate UGIs while getting the TaskUmbilicalProtocol proxy and
  5360. while executing the task. (Jakob Homan via ddas)
  5361. MAPREDUCE-1399. The archive command shows a null error message (nicholas
  5362. via mahadev)
  5363. MAPREDUCE-1305. Improve efficiency of distcp -delete. (Peter Romianowski
  5364. via cdouglas)
  5365. MAPREDUCE-1474. Update forrest documentation for Hadoop Archives. (Mahadev
  5366. Konar via cdouglas)
  5367. MAPREDUCE-1400. Use tr rather than sed to effect literal substitution in
  5368. the build script. (Allen Wittenauer via cdouglas)
  5369. MAPREDUCE-1358. Avoid false positives in OutputLogFilter. (Todd Lipcon via
  5370. cdouglas)
  5371. MAPREDUCE-1490. Fix a NullPointerException that could occur during
  5372. instantiation and initialization of the DistributedRaidFileSystem.
  5373. (Rodrigo Schmidt via dhruba)
  5374. MAPREDUCE-1476. Fix the M/R framework to not call commit for special
  5375. tasks like job setup/cleanup and task cleanup.
  5376. (Amareshwari Sriramadasu via yhemanth)
  5377. MAPREDUCE-1398. Fix TaskLauncher to stop waiting for slots on a TIP that
  5378. is killed / failed.
  5379. (Amareshwari Sriramadasu via yhemanth)
  5380. MAPREDUCE-1491. The parity files created by the RAID are combined
  5381. using Hadoop Archive Files (HAR). (Rodrigo Schmidt via dhruba)
  5382. MAPREDUCE-1378. URL encode link in jobhistory.jsp to avoid errors caused by
  5383. unescaped characters. (E. Sammer via cdouglas)
  5384. MAPREDUCE-1519. RaidNode fails to create new parity file
  5385. if an older version already exists. (Rodrigo Schmidt via dhruba)
  5386. MAPREDUCE-1537. Fixes a compilation problem in a testcase after commit
  5387. HDFS-984. (Jitendra Nath Pandey via ddas)
  5388. MAPREDUCE-1537. The patch makes the job client call the getDelegationToken
  5389. only when security is enabled. (Jitendra Nath Pandey via ddas)
  5390. MAPREDUCE-1510. RAID should regenerate parity files if they get deleted.
  5391. (Rodrigo Schmidt via dhruba)
  5392. MAPREDUCE-1421. Fix the LinuxTaskController tests failing on trunk after
  5393. the commit of MAPREDUCE-1385. (Amareshwari Sriramadasu via vinodkv)
  5394. MAPREDUCE-1520. Fix TestMiniMRLocalFS failure caused by regression in
  5395. getting user working dir. (Amareshwari Sriramadasu via cdouglas)
  5396. MAPREDUCE-1512. RAID uses HarFileSystem directly instead of
  5397. FileSystem.get (Rodrigo Schmidt via dhruba)
  5398. MAPREDUCE-1435. Fix symlink handling in task work directory when
  5399. cleaning up, essentially to avoid following links.
  5400. (Ravi Gummadi via yhemanth)
  5401. MAPREDUCE-1518. RaidNode does not run the deletion check on the
  5402. directory that stores the parity files. (Rodrigo Schmidt via dhruba)
  5403. MAPREDUCE-1573. TestStreamingAsDifferentUser fails if run as tt_user.
  5404. (Ravi Gummadi via vinodkv)
  5405. MAPREDUCE-927. Cleanup of task-logs should happen in TaskTracker instead
  5406. of the Child. (Amareshwari Sriramadasu via vinodkv)
  5407. MAPREDUCE-1578. Decouple HadoopArchives vesrion from HarFileSystem version.
  5408. (Rodrigo Schmidt via szetszwo)
  5409. MAPREDUCE-1422. Fix cleanup of localized job directory to work if files
  5410. with non-deletable permissions are created within it.
  5411. (Amar Kamat via yhemanth)
  5412. MAPREDUCE-1306. Randomize the arrival of heartbeat responses in Mumak.
  5413. (Tamas Sarlos via cdouglas)
  5414. MAPREDUCE-1579. archive: check and possibly replace the space charater
  5415. in source paths. (szetszwo)
  5416. MAPREDUCE-1536. DataDrivenDBInputFormat does not split date columns correctly.
  5417. (Aaron Kimball via enis)
  5418. MAPREDUCE-890. After HADOOP-4491, the user who started mapred system is
  5419. not able to run job. (Ravi Gummadi via vinodkv)
  5420. MAPREDUCE-1615. Fix compilation of TestSubmitJob. (cdouglas)
  5421. MAPREDUCE-1508. Protect against NPE in TestMultipleLevelCaching. (Aaron
  5422. Kimball via cdouglas)
  5423. MAPREDUCE-1497. Suppress spurious findbugs warning about IndexCache
  5424. synchronization. (Amareshwari Sriramadasu via cdouglas)
  5425. MAPREDUCE-1420. Fix TestTTResourceReporting failure. (Scott Chen via
  5426. cdouglas)
  5427. MAPREDUCE-1480. Correctly initialize child RecordReaders in
  5428. CombineFileInputFormat. (Aaron Kimball via cdouglas)
  5429. MAPREDUCE-1348. Fix block forensics packaging. (Tom White via cdouglas)
  5430. MAPREDUCE-1628. HarFileSystem shows incorrect replication numbers and
  5431. permissions. (szetszwo via mahadev)
  5432. MAPREDUCE-1602. Fix the error message for the case that src does not
  5433. exist. (szetszwo)
  5434. MAPREDUCE-1585. Create Hadoop Archives version 2 with filenames
  5435. URL-encoded (rodrigo via mahadev)
  5436. MAPREDUCE-1523. Sometimes rumen trace generator fails to extract the job
  5437. finish time. (dick king via mahadev)
  5438. MAPREDUCE-1635. ResourceEstimator does not work after MAPREDUCE-842.
  5439. (Amareshwari Sriramadasu via vinodkv)
  5440. MAPREDUCE-889. binary communication formats added to Streaming by
  5441. HADOOP-1722 should be documented. (Klaas Bosteels via tomwhite)
  5442. MAPREDUCE-1031. ant tar target doens't seem to compile tests in contrib
  5443. projects. (Aaron Kimball via tomwhite)
  5444. MAPREDUCE-1692. Removed unused testcase TestStreamedMerge.
  5445. (Sreekanth Ramakrishnan and Amareshwari Sriramadasu via yhemanth)
  5446. MAPREDUCE-1062. Fix ReliabilityTest to work with retired jobs. (Sreekanth
  5447. Ramakrishnan via cdouglas)
  5448. MAPREDUCE-1409. IOExceptions thrown from FIleOutputCommitter::abortTask
  5449. should cause the task to fail. (Amareshwari Sriramadasu via cdouglas)
  5450. MAPREDUCE-1695. Include capacity scheduler in findbugs and javadoc-dev
  5451. targets and also fix existing warnings. (Hong Tang via yhemanth)
  5452. MAPREDUCE-1494. Ensure TestJobDirCleanup verifies the correct paths.
  5453. (Amareshwari Sriramadasu via cdouglas)
  5454. MAPREDUCE-1622. Include missing slf4j dependencies. (cdouglas)
  5455. MAPREDUCE-1515. Accept java5.home from build.properties, not only from the
  5456. command line when building forrest docs. (Al Thompson via cdouglas)
  5457. MAPREDUCE-1618. Add missing javadoc to JobStatus::*JobAcls. (Amareshwari
  5458. Sriramadasu via cdouglas)
  5459. MAPREDUCE-1219. Remove job level metrics from jobtracker metrics to ease
  5460. undue load on jobtracker. (Sreekanth Ramakrishnan via sharad)
  5461. MAPREDUCE-1604. Add Forrest documentation for Job ACLs.
  5462. (Amareshwari Sriramadasu via yhemanth)
  5463. MAPREDUCE-1705. Archiving and Purging of HDFS parity files should
  5464. handle globbed policies accurately. (Rodrigo Schmidt via dhruba)
  5465. MAPREDUCE-1612. job conf file is not accessible from job history web page.
  5466. (Ravi Gummadi and Sreekanth Ramakrishnan via vinodkv)
  5467. MAPREDUCE-1397. NullPointerException observed during task failures.
  5468. (Amareshwari Sriramadasu via vinodkv)
  5469. MAPREDUCE-1728. Oracle timezone strings do not match Java.
  5470. (Aaron Kimball via tomwhite)
  5471. MAPREDUCE-1609. TaskTracker.localizeJob should not set permissions on
  5472. job log directory recursively. (Amareshwari Sriramadasu via vinodkv)
  5473. MAPREDUCE-1657. After task logs directory is deleted, tasklog servlet
  5474. displays wrong error message about job ACLs. (Ravi Gummadi via vinodkv)
  5475. MAPREDUCE-1727. TestJobACLs fails after HADOOP-6686. (Ravi Gummadi via vinodkv)
  5476. MAPREDUCE-1611. Refresh nodes and refresh queues doesnt work with service
  5477. authorization enabled. (Amar Kamat via vinodkv)
  5478. MAPREDUCE-1276. Correct flaws in the shuffle related to connection setup
  5479. and failure attribution. (Amareshwari Sriramadasu via cdouglas)
  5480. MAPREDUCE-1372. ConcurrentModificationException in JobInProgress.
  5481. (Dick King and Amareshwari Sriramadasu via tomwhite)
  5482. MAPREDUCE-118. Fix Job.getJobID(). (Amareshwari Sriramadasu via sharad)
  5483. MAPREDUCE-913. TaskRunner crashes with NPE resulting in held up slots,
  5484. UNINITIALIZED tasks and hung TaskTracker. (Amareshwari Sriramadasu and
  5485. Sreekanth Ramakrishnan via vinodkv)
  5486. MAPREDUCE-1725. Fix MapReduce API incompatibilities between 0.20 and 0.21.
  5487. (tomwhite)
  5488. MAPREDUCE-1606. TestJobACLs may timeout as there are no slots for launching
  5489. JOB_CLEANUP task. (Ravi Gummadi via vinodkv)
  5490. MAPREDUCE-1765. Correct streaming documentation for StreamXmlRecordReader.
  5491. (Corinne Chandel via amareshwari)
  5492. MAPREDUCE-1880. Fix BigDecimal.divide(..) in the pi example. (szetszwo)
  5493. MAPREDUCE-1885. Revert FileSystem create method that takes CreateFlags
  5494. (MapReduce part of HADOOP-6826). (Ravi Gummadi via tomwhite)
  5495. MAPREDUCE-1870. Harmonize MapReduce JAR library versions with Common and
  5496. HDFS. (tomwhite)
  5497. MAPREDUCE-1791. Remote cluster control functionality needs JavaDocs
  5498. improvement (Konstantin Boudnik)
  5499. MAPREDUCE-1942. 'compile-fault-inject' should never be called directly.
  5500. (Konstantin Boudnik)
  5501. MAPREDUCE-1876. Fixes TaskAttemptStartedEvent to correctly log event type
  5502. for all task types. (Amar Kamat via amareshwari)
  5503. MAPREDUCE-1926. MapReduce distribution is missing build-utils.xml.
  5504. (tomwhite)
  5505. MAPREDUCE-2012. Some contrib tests fail in branch 0.21 and trunk.
  5506. (Amareshwari Sriramadasu via tomwhite)
  5507. MAPREDUCE-1980. Fixes TaskAttemptUnsuccessfulCompletionEvent and
  5508. TaskAttemptFinishedEvent to correctly log event type for all task types.
  5509. (Amar Kamat via amareshwari)
  5510. MAPREDUCE-1856. Extract a subset of tests for smoke (DOA) validation (cos)