hadoop/mapreduce/CHANGES.txt

Hadoop MapReduce Change Log

Trunk (unreleased changes)


    MAPREDUCE-279

    Fixing scheduling deadlock in AM because of incorrect headRoom values from
    RM. The bug happens when AM releases containers and RM decrements current
    memory usage twice for all those containers. (vinodkv)

    Disabling locality-wait in CapacityScheduler for now to prevent uber-slow
    scheduling for apps with no data-locality constraints (sleep-job like). (vinodkv)

    Fix to schedule reduces irrespective of the headroom when all maps are
    done so as to avoid stall in reduce-scheduling when slow-start is
    disabled. (Sharad Agarwal via vinodkv).

    Fix RM app start/finish time and diagnostics. (llu)

    Fix race condition between multiple localizers on a single node. (cdouglas via mahadev)

    Improve logging for AM when requesting containers to show the right ask 
    and release fields (mahadev)

    Fix NPE when requesting attempts for completed jobs. (Siddharth Seth via llu)

    Changes a couple of usages of FileContext to FileSystem in TaskAttemptImpl 
    to handle distributed cache path resolutions on non-default filesystems.
    (ddas)

    Hack until MAPREDUCE-2365 is fixed to make PIG work with MRV2. (mahadev)

    Bug fix to set correct state on containers so as to avoid duplicate
    containers from RM to AM. (vinodkv)

    Adding attempts page to the history server UI. (vinodkv)

    Task diagnostic info made available on the AM UI. (vinodkv)

    MAPREDUCE-2598. Fix NPE and UI for JobHistory. (Siddharth Seth via llu)

    Fix NPE when killing/failing already killed/failed tasks. (llu)
   
    Update install instructions with svn unsplit (Thomas Graves via mahadev)

    MAPREDUCE-2587. Generate yarn version for UI. (Thomas Graves via llu)

    Fix stopContainer for setsid challenged platforms. (llu)

    Fix concurrent modification exception in the Capacity Scheduler (mahadev)

    Changes for invoking rack resolution in the RM and in the AM (ddas)

    Fix ClassCastException in JobHistoryServer for certain jobs.
    (Siddharth Seth via llu)

    MAPREDUCE-2569. Ensure root queue allocated 100% capacity. (Jonathan
    Eagles via cdouglas)

    Fix NPE in history event handling (siddharth seth via mahadev)

    Job level node blacklisting. (sharad)

    Fixing the bug which was causing FAILED jobs to be displayed as COMPLETED
    on the RM UI. (vinodkv)

    Fixing broken link to logs for container on NM web UI. (vinodkv)

    Miscellaneous UI fixes + source code formatting for
    MR JobHistoryEventHandler. (vinodkv)

    Display failed/killed attempts of the task on MR AM UI
    separately. (vinodkv)

    Changes a couple of usages of FileContext to FileSystem in YarnRunner
    to handle distributed cache path resolutions on non-default filesystems.
    (ddas)

    Fix to exclude images dir into the tar distribution (luke lu via gkesavan)

    Fix more rounding off problems in reduce ramp up. Also fix a bug preventing
    the application of the cap on reduce ramp-up. (Sharad Agarwal via vinodkv)

    Fix rounding off problem in reduce ramp up. (sharad)

    MAPREDUCE-2582. Cleanup JobHistory event generation.(Siddharth Seth via sharad)   
 
    Add ability to includes src files in assembly target for maven (Luke Lu via mahadev)
 
    Added few job diagnostic messages. (sharad)

    Disable Uber AM. (mahadev)

    Intermittent TestMRApp failures on faster Linux desktop (part 2) (Luke lu via mahadev)
    
    Including source files in release distribution (luke lu via mahadev)

    Intermittent TestMRApp failures on faster Linux desktop. (Luke lu via mahadev)

    Added clover in pom dependency. (sharad)

    Sending Job diagnostics from AM to RM and redirect to history-url on job
    completion. (vinodkv and sharadag)

    Fix classpath construction for Task. (vinodkv via sharad)

    Remove retries in dist cache so that NM's do not shutdown (mahadev)

    Solving NPEs during ContainerManager#StopContainer. Also removing
    the unused ContainerManager#CleanupContainer api. (vinodkv)

    Fixed deadlock during expiring NMs. (acmurthy)

    Re-enabling Uber-AM feature. (vinodkv)
 
    Fully resolve paths when launching containers. (Siddharth Seth)
   
    MAPREDUCE-2566. YarnConfiguration should reloadConfiguration if 
    instantiated with a non YarnConfiguration object. (siddharth seth)

    Work around broken signaling in public cache. (cdouglas)

    Cleanup redundant code in TaskAttemptImpl. (sharad)

    Disable aggregation of logs onto DFS till JobHistoryServer starts
    serving logs. (vinodkv)

    Add deletion of distributed cache resources. (cdouglas)

    Fix class cast exception in Task abort for old mapreduce apis. (sharad)

    Fix various issues with Web UI's. (Luke Lu)

    Fixed inconsistency in QueueACL enums. (acmurthy)

    Fixing the wrong config key used in JobHistory that prevented configuring
    move-thread interval. (vinodkv)

    MAPREDUCE-2539. Fixed NPE in getMapTaskReports in JobClient. (Robert Evans via
    acmurthy) 

    MAPREDUCE-2531. Fixed jobcontrol to downgrade JobID. (Robert Evans via
    acmurthy) 

    Support for min and max container capacity. (acmurthy and sharad)

    Re-enabled TestCapacityScheduler. (acmurthy)

    Fix logging for showing the state of job (FAILED/KILLED/SUCCEEDED) when it 
    completes (mahadev)

    MAPREDUCE-2537. The RM writes its log to 
    yarn-mapred-resourcemanager-<RM_Host>.out (Robert Evans via mahadev)

    Fix a corner case in headroom computation - now reservations are taken
    into account and headroom is computed much later to account for
    allocations/reservations. (acmurthy) 

    Fix to report job status if the application is KILLED/FAILED. (mahadev)

    Disable Job acls until fixed (mahadev)

    Fix container size rounding in AM and headroom in RM. (acmurthy and 
    sharad) 

    More cleaning up constants, removing stale code, and making conspicuous
    the envs that apps depend on to be provided by YARN. (vinodkv)

    Fix calculation of max-capacity for a queue, also fixed a bug in 
    registration of NodeManagers. (acmurthy) 
  
    Making each node aggregate all its user-logs to a separate hdfs
    file. (vinodkv)

    Fix for NPE in TestNMExpiry (siddharth seth)

    Fix another NPE in TestRMNMRPCResponseId. (mahadev)

    NodeStatus.getNodeHealthStatus().setBlah broken (siddharth seth)

    Fix a deadlock in the resourcemanager. (mahadev)
   
    Fix for NPE in TestRMNMRPCResponseId. (mahadev)

    Fixes in the handling of KILL events in the SUCCEEDED state for tasks
    in the application master (ddas)
    
    Fix build issue for using yarn.version instead of hadoop-mapred.version
    (mahadev and giri)
    
    Fixed TestFifoScheduler. (acmurthy)
    
    Fix reduce slow start. (sharad)
    
    MAPREDUCE-2552. Fixed NPE in CompletedJob in JobHistoryServer.
    (Siddharth Seth via acmurthy)
     
    MAPREDUCE-2551. Added JobSummaryLog. (Siddharth Seth via acmurthy)
     
    Added ability to decommission nodes and completed RM administration tools
    to achieve parity with JobTracker. (acmurthy) 

    Allowing hdfs calls from streaming/pipes tasks. (vinodkv)

    Reduce ramp up and zero maps support. (sharad)

    Adding some more logging for AM expiry logs (mahadev)

    Ensure 'lost' NodeManagers are dealt appropriately, the containers are
    released correctly. (acmurthy) 

    Making pipes work with YARN. Changed pipes to get log-locations from an
    environmental variable. (vinodkv)

    Fixed NPE in CS by checking Application state before scheduling and fixing
    synchronization in CS. (acmurthy)

    Fixing NPE on speculator in MRAppMaster and making job-history optional in
    tests to make test goal succeed. (vinodk and sharadag).

    Cleaning up configuration constants in mapreduce modules. (vinodkv)

    Fixing and reneabling TestContainerTokenSecretManager. (vinodkv)

    Set correct version of avro-maven-plugin that is available in
    apache maven repositories. (vinodkv)
  
    Fixes for TestFail/Kill (ddas)
    
    First fix for making basic speculative execution work (ddas)

    Bugfix for using user staging directory for history files (siddharth seth via mahadev)

    MAPREDUCE-2536. Backporting changes to MR-279.
   
    Making streaming -file option work. Also minor fixes for successful
    compilation of contrib tests. (vinodkv)

    Fix for public dist cache to work with non default hdfs (mahadev &ddas)

    Fixed an NPE during handling of unnecessary reservations in CS. (acmurthy)

    MAPREDUCE-2532. Metrics for NodeManager (Luke Lu via mahadev)

    MAPREDUCE-2527. Metrics for MRAppMaster (Luke lu via mahadev)

    Add debug config for delaying delete of local files. (cdouglas)

    Fixing race condition leader to hung jobs in scheduler negotiator (mahadev)

    Add debug statements for AM not launching (mahadev)

    Adding job kill for any state that the job is in with access control. (mahadev)

    Added acl check for RMAdmin. (acmurthy)

    Made number of RPC server threads configurable. (acmurthy) 

    Add public cache. (cdouglas)

    MAPREDUCE-2534. Fix CI breaking hard coded version in jobclient pom. 
    (Luke Lu via mahadev)

    MAPREDUCE-2522. Security for JobHistory service. (Siddharth Seth via 
    mahadev)

    Added metrics for tracking reservations in CapacityScheduler. (Luke Lu via
    acmurthy)

    Fix queue refresh to correctly record newly added queues in
    CapacityScheduler. (acmurthy)

    Fix TestRuntimeEstimators (Siddharth Seth via ddas)

    Ensure unused containers released by AM are correctly counted for
    queue-capacity. (acmurthy)

    Fix distributed-cache related bugs. (vinodkv)

    Fix for regression on the scheduling of reduces before maps are done (ddas)

    Fix NPE in test case (mahadev)

    Use DefaultContainerExecutor for integration tests. (cdouglas)

    Fix nodemanager expiry to not throw OOM. (mahadev)

    Changed Scheduler to return available limit to AM in the allocate api.
    (acmurthy)

    Refactored RMContainerAllocator to release unused containers. (sharad) 

    Fix null pointer exception in kill task attempt (mahadev)

    Fix for the reduces getting ahead of maps in the scheduling (ddas)    

    Distributed cache bug fix to pass Terasort. (vinodkv)

    Add junit jar to lib in assembly (mahadev and luke)

    MAPREDUCE-2509. Fix NPE in UI for pending attempts. (luke lu via mahadev) 

    Fix job hang if the AM launch fails. (mahadev) 
   
    MAPREDUCE-2504. race in JobHistoryEventHandler stop (siddharth seth via mahadev)

    MAPREDUCE-2500. PB factories are not thread safe (siddharth seth via mahadev) 

    Fix the tests to use jvm fork mode to avoid errors in shutting down
    services (sidharth seth)

    Fixed Composite Service to shutdown services if an error occurs on 
    starting any one of those (mahadev & chris)

    Fixing a bug to do with setting the staging dir. (ddas)

    Fixing a bug in previous patch (r1103657). Now bin/yarn truly shouldn't be
    launched multiple times in a single AM. (vinodkv)

    Make logging and memory for AM configurable for the user via command 
    line (mahadev) 

    Makes uber-task disabled by default (ddas)
    
    Fix CapacityScheduler (LeafQueue) to not allocate DATA_LOCAL containers
    when they are not required on the rack. (acmurthy)

    Fix CapacityScheduler to release unused reservations on application
    completion. (acmurthy)

    Launching bin/yarn and bin/mapred only *once* in AM for constructing
    classpaths to avoid multiple forks and huge vmem usage by AM. (vinodkv)

    Fix calculation of maximum capacity in ParentQueue to use its parent's 
    absolute-capacity rather than its own absolute-capacity. (acmurthy)

    Fixing a bug in JobIDPbImpl that's causing AM to crash randomly. (vinodkv)

    Fix calculation of maximum capacity to use parent's absolute-capacity
    rather than the leaf queue's absolute-capacity. (acmurthy)

    Speed up communication between MR AM and RM by relying on a new config
    rather than AM_EXPIRY_INTERVAL which is too large. (acmurthy)

    Fix for clearing container requests on an AM failure and add tostring methods
    to taskids and taskattemptids for better grep support. (mahadev)
   
    Support fail-fast for MR jobs. (ddas)
   
    Support mapreduce old (0.20) APIs. (sharad)

    Fixed reservation's bad interaction with delay scheduling in CS.
    (acmurthy)

    Fixed CS user limits. (acmurthy)

    Adding valid state to ASM on a finish when its already completed and 
    also disble UberAM. (mahadev)

    Fix a race in MR task that was causing MR containers to overwrite each
    other's job.xml. Also fix leaking attempt-dirs in app-local-dir. (vinodkv)

    Minor fix for install instructions. (mahadev)
    
    Add license header and minor cleanup in history server. (Siddharth Seth 
    via sharad)

    Removal of stale application-log dirs from NM local disks. (vinodkv)

    Fix TestQueueMetrics. (Luke Lu via sharad)

    Fix assembly to add mapreduce shell scripts to the assembly package. (mahadev)
    
    Propagate error back to client in case of a job submission failure (mahadev)
    
    MAPREDUCE-2480: MR App should not depend on hard coded version of shuffle (luke lu via mahadev)

    Fix NM to use multiple disks for local files and the userlogs. (vinodkv)

    Improved TestJobHistoryEvents and TestJobHistoryParsing. (sharad)

    MAPREDUCE-2478. Improve history server. (Siddharth Seth via sharad)

    Fix Null Pointer in TestUberAM. (sharad) 

    Fix refreshProxy in ClientServiceDelegate. (sharad)

    Client reconnect to restarted AM. (sharad)

    Replacing FileContext usage with FileSystem to work around security authentication
    issues with FileContext against a secure DFS. (vinodkv)

    Fixing three tight-loops in RM that are causing high cpu-usage. (vinodkv)

    Adding user log handling for YARN. Making NM put the user-logs on DFS and providing log-dump tools. (vinodkv)

    MAPREDUCE-2468. Add metrics for NM Shuffle. (Luke Lu via cdouglas)

    Completing RM Restart. Completed Phase 3 of making sure events are logged and restored (mahadev)

    Moving userlogs out of container work-dir into a separate directory structure. (vinodkv)

    Implement Job Acls in MR Application Master. (sharad)
    
    Implement 'delay scheduling' for better locality in CapacityScheduler and
    improved high-ram applications. (acmurthy)

    MAPREDUCE-2462. Write job conf along with JobHistory, other minor improvements. 
    (Siddharth Seth via sharad)

    Fix to send finish application event only when the application is finished (mahadev)

    RM Restart Phase 2 - Completed the recovery of components in the RM (mahadev)

    MAPREDUCE-2434. Metrics for ResourceManager. (Luke Lu via acmurthy)

    Fix container launch w/ inconsistent credential file naming. (cdouglas)

    Disable ContainerMonitoring for non-linux systems. (vinodkv)

    Fix file creation in JobHistoryEventHandler. (sharad)

    Reorient container localization to be per-container rather than
    per-application. (cdouglas)

    Completing the ZooKeeper Store for ResourceManager state. (mahadev)

    Added support High-RAM applications in CapacityScheduler. (acmurthy) 

    Recovery of MR Application Master from failures. (sharad)
   
    Add ACLs for queues and command-line utilities for viewing them.
    (acmurthy)

    Add Containers' logs' view to NM UI and link it from AM UI. (vinodkv)

    Fix race condition in TestJobHistoryEvents and TestJobHistoryParsing. (sharad)
    
    Introducing web-UI for NodeManager and linking it from RM UI. (vinodkv)

    Added functionality to stop/start queues. (acmurthy)
    
    Added functionality to refresh queues at runtime via the 'bin/yarn
    rmadmin' command. (acmurthy) 
    
    Fixed computation of user-limits at runtime. (acmurthy) 
    
    Implement 'bin/mapred queue [-info [-showJobs]] [-list] and enhanced 
    'bin/mapred job -list' to show queue and ApplicationMaster information. 
    (acmurthy)
    
    Implement 'bin/mapred job -list' and 'bin/mapred job
    -list-active-trackers'. (acmurthy)

    Add HistoryCleanerService to Job History server. (Krishna Ramachandran
    via sharad)

    Install sanitized poms for downstream sanity (Luke Lu via mahadev)

    WebApp for Job History (Krishna Ramachandran via mahadev)
  
    Fix YarnRemoteException to give more details. (siddharth seth via mahadev)

    MAPREDUCE-2405. Implement uber-AppMaster (in-cluster LocalJobRunner for
    MRv2). (Greg Roelofs and Sharad Agarwal via mahadev)

    MAPREDUCE-2424. Polish uber-AppMaster: add uber-AM counters and GUI
    indicators.  (Greg Roelofs via mahadev)
   
    Add fail count to the command line of the application master. (mahadev)

    MAPREDUCE-2440. Name clashes in TypeConverter (luke via mahadev)
  
    Implementing Containers' memory monitoring. (vinodkv)

    MAPREDUCE-2405: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2) (Greg Roelofs via mahadev)

    Implement health-checks for the node - server side(ResourceManager) changes. (vinodkv)

    MAPREDUCE-2414. Change MRv2 to use generic interfaces. (Siddharth Seth via acmurthy)

    Implement restart for resource manager phase 1 - Helper classes to store and restore the data structures. (mahadev)

    MAPREDUCE-2433. YARNApplicationConstants hard code app master jar version (Luke Lu via mahadev)
 
    Clear application notification if sent once to NodeManager (mahadev)

    MAPREDUCE-2458. Rename sanitized pom.xml in build directory to work around IDE bug 
    (luke lu via mahadev)

  INCOMPATIBLE CHANGES

    MAPREDUCE-1866. Removes deprecated class
    org.apache.hadoop.streaming.UTF8ByteArrayUtils. (amareshwari)

    MAPREDUCE-1664. Changes the behaviour of the combination of job-acls
    when they function together with queue-acls. (Ravi Gummadi via vinodkv)

  NEW FEATURES

    MAPREDUCE-1804. Stress-test tool for HDFS introduced in HDFS-708.
    (Joshua Harlow via shv)

    MAPREDUCE-220. Collect cpu and memory statistics per task. (Scott Chen via
    acmurthy)

  IMPROVEMENTS

    MAPREDUCE-2140. Regenerate fair scheduler design doc PDF. (matei)

    MAPREDUCE-1546. Redirect all job pages to corresponding history page
    if job is not in JT memory. (Scott Chen via sharad)

    MAPREDUCE-1092. Enable assertions for unit tests. (Eli Collins via
    cdouglas)

    MAPREDUCE-1680. Add a metric recording JobTracker heartbeats processed.
    (Dick King via cdouglas)

    MAPREDUCE-1761. FairScheduler allows separate configuration of node
    and rack locality wait time (Scott Chen via dhruba)

    MAPREDUCE-1539. authorization checks for inter-server protocol
    (based on HADOOP-6600) (Boris Shkolnik via shv)

    MAPREDUCE-1798. Names the configuration keys for the Kerberos
    principals better. (Boris Shkolnik via ddas)

    MAPREDUCE-1773. streaming doesn't support jobclient.output.filter.
    (Amareshwari Sriramadasu via vinodkv)

    MAPREDUCE-1785. Add streaming config option for not emitting the key.
    (Eli Collins via sharad)

    MAPREDUCE-572. If #link is missing from uri format of -cacheArchive
    then streaming does not throw error. (Amareshwari Sriramadasu via
    vinodkv)

    MAPREDUCE-1545. Add timestamps for first task type launched in job summary.
    (Luke Lu via cdouglas)

    MAPREDUCE-1543. Add an audit log for authentication events. (Amar Kamat and
    Luke Lu via cdouglas)

    MAPREDUCE-1762. Add ability to set values of task counters. (Scott Chen via
    cdouglas)

    MAPREDUCE-1533. Reduce overhead of logging and string manipulation during
    heartbeat processing. (Amar Kamat and Dick King via cdouglas)

    MAPREDUCE-1516. JobTracker issues delegation tokens only if the user's
    authentication is Kerberos. (Jitendra Pandey via ddas)

    MAPREDUCE-647. Update distcp forrest documentation to reflect the changes
    of HADOOP-5472, MAPREDUCE-642 and HADOOP-5620.  (Rodrigo Schmidt via
    szetszwo)

    MAPREDUCE-1851. Documents configuration parameters in streaming.
    (amareshwari)
    
    MAPREDUCE-1868. Add a read and connection timeout to JobClient while
    pulling tasklogs. (Krishna Ramachandran via acmurthy)

    MAPREDUCE-1788. Ensure failure to setup CompletedJobStatusStore is not
    silently ignored by the JobTracker. (Krishna Ramachandran via acmurthy)

    MAPREDUCE-1850. Includes job submit host information (name and ip) in
    jobconf and jobdetails display (Krishna Ramachandran via amareshwari)

    MAPREDUCE-1893. Slive with multiple reducers. (shv)

    MAPREDUCE-1248. Fixes redudant memory copying in StreamKeyValUtil.
    (Ruibang He via amareshwari)

    MAPREDUCE-1840. Enhancements to Gridmix benchmark simulating user
    diversity, queue replay, and task duration for JobTracker load testing.
    Also includes compatibility with security enhancements, and scalability
    improvements. (Amar Kamat, Rahul Singh, Hong Tang, and cdouglas)

    MAPREDUCE-1848. Put number of speculative, data local, rack local 
    tasks in JobTracker metrics. (Scott Chen via dhruba)

    MAPREDUCE-1935. Makes the Distcp to work in a secure environment.
    (Boris Shkolnik via ddas)

    MAPREDUCE-1945. The MapReduce component for HADOOP-6632.
    (Kan Zhang & Jitendra Pandey via ddas)

    MAPREDUCE-1936. Modify Gridmix3 to support more tunable parameters for
    stress submission and sleep jobs. (Hong Tang via cdouglas)

    MAPREDUCE-1733. Makes pipes applications secure. (Jitendra Pandey via ddas)

    MAPREDUCE-1566. Adds a configuration attribute using which job clients can
    specify a credentials file. The tokens from there will be passed to the job.
    (Jitendra Pandey and Owen O'Malley via ddas)

    MAPREDUCE-1624. Documents the job credentials and associated details to do 
    with delegation tokens (on the client side). 
    (Jitendra Pandey and Devaraj Das via ddas)

    MAPREDUCE-1834. TestSimulatorDeterministicReplay timesout on trunk.
    (Hong Tang via mahadev)

    MAPREDUCE-1993. Fixes test failure
    TestTrackerDistributedCacheManagerWithLinuxTaskController. (Devaraj Das
    via amareshwari)

    MAPREDUCE-1523. Making Mumak work with Capacity-Scheduler (Anirban Das
    via mahadev)

    MAPREDUCE-1920. Enables completed jobstatus store by default. (Tom White
    via amareshwari)

    MAPREDUCE-1881. Improve TaskTrackerInstrumentation to enable collection of
    advanced metrics. (Matei Zaharia via acmurthy)

    MAPREDUCE-1548. Hadoop archives preserve times and other properties from 
    original files. (Rodrigo Schmidt via dhruba)

    MAPREDUCE-1517. Supports streaming job to run in the background. (Bochun Bai
    via amareshwari)

    MAPREDUCE-1819. RaidNode is now smarter in submitting Raid jobs. (Ramkumar
    Vadali via schen)

  OPTIMIZATIONS

    MAPREDUCE-1354. Enhancements to JobTracker for better performance and
    scalability. (Arun C. Murthy & Richard King via acmurthy) 

    MAPREDUCE-1829. JobInProgress.findSpeculativeTask should use min() to
    find the candidate instead of sort(). (Scott Chen via vinodkv)

  BUG FIXES

    MAPREDUCE-1845. FairScheduler.tasksToPreempt() can return negative number.
    (Scott Chen via matei)

    MAPREDUCE-1707. TaskRunner can get NPE in getting ugi from TaskTracker.
    (Vinod Kumar Vavilapalli)
    
    MAPREDUCE-1532. Ensures that delegation tokens is obtained as the
    actual user when the proxy-user is used for submitting jobs. Also
    refactors the DelegationTokenToRenew class. (ddas)

    MAPREDUCE-1558. Fixes MRAdmin to look up the principal of the 
    JobTracker and use that in the RefreshUserToGroupsMapping protocol and
    RefreshAuthorizationPolicyProtocol. (Boris Shkolnik via ddas)

    MAPREDUCE-1662. Remove unused methods from TaskRunner. (Amareshwari
    Sriramadasu via cdouglas)

    MAPREDUCE-1617. Use IPv4 stack for unit tests. (Amar Kamat and Luke Lu via
    cdouglas)

    MAPREDUCE-1599. Fixes MRBench so that it reuses tokens across jobs
    correctly. (Jitendra Nath Pandey via ddas)

    MAPREDUCE-1836. Refresh for proxy superuser config (mr part for HDFS-1096).
    (Boris Shkolnik via shv)

    MAPREDUCE-1505. Create RPC client on job submission, not in cstr of Job
    instance. (Dick King via cdouglas)

    MAPREDUCE-1813. NPE in PipeMapred.MRErrorThread. (Ravi Gummadi via vinodkv)

    MAPREDUCE-1225. Fixes DistributedCache to check if the file is fresh or not,
    for the first localization also. (Zhong Wang via amareshwari)

    MAPREDUCE-1559. Fixes the token renewer to use the JobTracker's 
    credentials for talking to the NameNode. (ddas)

    MAPREDUCE-1492. Delete  obsolete har files used on the parity files
    of hdfs raid. (Rodrigo Schmidt via dhruba)

    MAPREDUCE-1857. Removes unused configuration parameters in streaming.
    (amareshwari)

    MAPREDUCE-1887. MRAsyncDiskService now properly absolutizes volume root
    paths. (Aaron Kimball via zshao)

    MAPREDUCE-1863. Fix NPE in Rumen when processing null CDF for failed task
    attempts. (Amar Kamat via cdouglas)

    MAPREDUCE-1864. Removes uninitialized/unused variables in
    org.apache.hadoop.streaming.PipeMapRed. (amareshwari)

    MAPREDUCE-1888. Fixes Streaming to override output key and value types, 
    only if mapper/reducer is a command. (Ravi Gummadi via amareshwari)

    MAPREDUCE-577. Fixes duplicate records in StreamXmlRecordReader.
    (Ravi Gummadi via amareshwari)

    MAPREDUCE-1894. Fixed a bug in DistributedRaidFileSystem.readFully() 
    that was causing it to loop infinitely. (Ramkumar Vadali via dhruba)

    MAPREDUCE-1838. Reduce the time needed for raiding a bunch of files
    by randomly assigning files to map tasks. (Ramkumar Vadali via dhruba)

    MAPREDUCE-1820. Fix InputSampler to clone sampled keys. (Alex Kozlov via
    cdouglas)

    MAPREDUCE-1528. Incorporates the changes to the credentials API done in
    HADOOP-6845. Also, introduces Credentials in JobConf, and in JobContext.
    (Jitendra Pandey and Arun Murthy via ddas)

    MAPREDUCE-1865. Rumen should also support jobhistory files generated using
    trunk. (Amar Kamat via amareshwari)

    MAPREDUCE-1621. Fixes NPE in TextOutputReader.getLastOutput if it has never
    read any output. (amareshwari)

    MAPREDUCE-1911. Fixes errors in -info message in streaming. (amareshwari) 

    MAPREDUCE-1772. Corrects errors in streaming documentation in forrest.
    (amareshwari)

    MAPREDUCE-1925. Fix failing TestRumenJobTraces. (Ravi Gummadi via cdouglas)

    MAPREDUCE-1718. Fixes a bug in the construction of jobconf key for the
    mapping that the tasks use at runtime for looking up delegation tokens.
    (Boris Shkolnik via ddas)

    MAPREDUCE-1701. Fixes a problem to do with exception handling in 
    delegation-token renewals. (Boris Shkolnik via ddas)

    MAPREDUCE-1686. Fixes StreamUtil.goodClassOrNull to find classes without
    package names. (Paul Burkhardt via amareshwari)

    MAPREDUCE-1288. Fixes TrackerDistributedCacheManager to take into account
    the owner of the localized file in the mapping from cache URIs to
    CacheStatus objects. (ddas)

    MAPREDUCE-1982. Fixes Rumen's TraceBuilder to extract job name from either 
    of configuration properties "mapreduce.job.name" and "mapred.job.name".
    (Ravi Gummadi via amareshwari)

    MAPREDUCE-1958. The MapReduce part corresponding to the HADOOP-6873.
    (Boris Shkolnik & Owen O'Malley via ddas)

    MAPREDUCE-1900. TaskTracker and JobTracker closes FileSystems, opened on
    behalf of users that it no longer requires. (Kan Zhang and ddas via ddas)

    MAPREDUCE-1992. Fixes a problem to do with bringing up the JobTracker in
    unsecure mode. (Kan Zhang via ddas)

    MAPREDUCE-1999. Fixes ClientProtocol to use the correct 
    DelegationTokenSelector. (Jitendra Pandey via ddas)

    MAPREDUCE-1780. AccessControlList.toString() is used for serialization of
    ACL in JobStatus.java. (Ravi Gummadi via vinodkv)

    MAPREDUCE-1961. Fix ConcurrentModificationException in Gridmix during
    shutdown. (Hong Tang via cdouglas)

    MAPREDUCE-2000. Fix parsing of JobHistory lines in Rumen when quotes are
    escaped. (Hong Tang via cdouglas)

    MAPREDUCE-2022. Fixes compilation errors in TestSubmitJob. (amareshwari)

    MAPREDUCE-1670. RAID policies should not scan their own destination path.
    (Ramkumar Vadali via dhruba)

    MAPREDUCE-1668. RaidNode Hars a directory only if all its parity files 
    have been created. (Ramkumar Vadali via dhruba)

    MAPREDUCE-2021. Fixes duplicate hostnames in CombineFileInputFormat's
    split locations. (amareshwari)

    MAPREDUCE-1375. Fixes flaky test TestFileArgs. (Todd Lipcon via
    amareshwari)

    MAPREDUCE-2023. TestDFSIO should not stop reading if curSize != bufferSize.
    (Hong Tang via szetszwo)

    MAPREDUCE-2031. Fixes test failures TestTaskLauncher and
    TestTaskTrackerLocalization. (Ravi Gummadi via amareshwari)

    MAPREDUCE-2046. Fixes CombineFileInputFormat to allow splits with size
    less than DFS block size. (dhruba borthakur via amareshwari)

    MAPREDUCE-1975. Fixes unnecessary InterruptedException log in gridmix.
    (Ravi Gummadi via amareshwari)

    MAPREDUCE-1597. Fixes CombineFileInputFormat to work with non-splittable
    files. (amareshwari)

    MAPREDUCE-2032. Fixes TestJobCleanup to cleanup test directory in
    tearDown. (Dick King via amareshwari)

    MAPREDUCE-1979. Fixes "Output directory already exists" error in gridmix
    when gridmix.output.directory is not defined. (Ravi Gummadi via
    amareshwari)

    MAPREDUCE-1918. Adds documentation to Rumen. (Amar Kamat via amareshwari)

    MAPREDUCE-2078. Fixes TraceBuilder to generate traces when a globbed job
    history path is given. (Amar Kamat via amareshwari)

    MAPREDUCE-1989. Fixes error message in gridmix when user resolver is set
    and no user list is given. (Ravi Gummadi via amareshwari)

    MAPREDUCE-2067.  Distinct minicluster services (e.g. NN and JT) overwrite
    each other's service policies.  (Aaron T. Myers via tomwhite)

    MAPREDUCE-2029. DistributedRaidFileSystem removes itself from FileSystem
    cache when it is closed. (Ramkumar Vadali via dhruba)

    MAPREDUCE-1816. HAR files used for RAID parity-bite have configurable 
    partfile size. (Ramkumar Vadali via dhruba)

    MAPREDUCE-2082. Fixes Pipes to create the jobtoken file in the right
    place. (Jitendra Pandey via ddas)

    MAPREDUCE-2095. Fixes Gridmix to run from compressed traces. (Ranjit
    Mathew via amareshwari)

    MAPREDUCE-1908. DistributedRaidFileSystem now handles ChecksumException
    correctly. (Ramkumar Vadali via schen)
    
    MAPREDUCE-2232. Add missing methods to TestMapredGroupMappingServiceRefresh.
    (Todd Lipcon via Eli Collins)

Release 0.21.1 - Unreleased

  NEW FEATURES

    MAPREDUCE-2040. Forrest Documentation for Dynamic Priority Scheduler.
    (Thomas Sandholm via tomwhite)

  BUG FIXES

    MAPREDUCE-1897. trunk build broken on compile-mapred-test (cos)

    MAPREDUCE-1280. Update Eclipse plugin to the new eclipse.jdt API.
    (Alex Kozlov via szetszwo)

    MAPREDUCE-1984. herriot TestCluster fails because exclusion is not there
    (Balaji Rajagopalan via cos)

    MAPREDUCE-2090. Clover build doesn't generate per-test coverage. (cos)

Release 0.21.0 - 2010-08-13

  INCOMPATIBLE CHANGES

    MAPREDUCE-516. Fix the starvation problem in the Capacity Scheduler 
    when running High RAM Jobs. (Arun Murthy via yhemanth)

    MAPREDUCE-358. Change org.apache.hadoop.examples. AggregateWordCount 
    and org.apache.hadoop.examples.AggregateWordHistogram to use new 
    mapreduce api. (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-245. Change Job and jobcontrol classes to use the List interface
    rather than ArrayList in APIs. (Tom White via cdouglas)

    MAPREDUCE-766. Enhanced list-blacklisted-trackers to display reasons
    for blacklisting a node. (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-817. Add a cache for retired jobs with minimal job info and 
    provide a way to access history file url. (sharad)

    MAPREDUCE-711. Moved Distributed Cache from Common to Map/Reduce
    project. (Vinod Kumar Vavilapalli via yhemanth)

    MAPREDUCE-895. Per the contract elucidated in HADOOP-6201, throw
    FileNotFoundException from FileSystem::listStatus rather than returning
    null. (Jakob Homan via cdouglas)

    MAPREDUCE-479. Provide full task id to map output servlet rather than the
    reduce id, only. (Jiaqi Tan via cdouglas)

    MAPREDUCE-873. Simplify job recovery. Incomplete jobs are resubmitted on 
    jobtracker restart. Removes a public constructor in JobInProgress. (sharad)

    HADOOP-6230. Moved process tree and memory calculator related classes from
    Common to Map/Reduce. (Vinod Kumar Vavilapalli via yhemanth)

    MAPREDUCE-157. Refactor job history APIs and change the history format to 
    JSON. (Jothi Padmanabhan via sharad)

    MAPREDUCE-849. Rename configuration properties. (Amareshwari Sriramadasu 
    via sharad)

    MAPREDUCE-1287. Only call the partitioner with more than one reducer.
    (cdouglas)

    MAPREDUCE-1385. Use the new UserGroupInformation from HADOOP-6299.
    (ddas via omalley)

    MAPREDUCE-1493. Authorization for job-history pages. (vinodkv)

    MAPREDUCE-1607. Task controller may not set permissions for a
    task cleanup attempt's log directory (Amareshwari Sriramadasu via vinodkv)

    MAPREDUCE-1683. Remove JNI calls from ClusterStatus cstr. (Arun Murthy and
    Luke Lu via cdouglas)

    MAPREDUCE-1855. Makes the refresh methods (for groups and proxy users) 
    independent of the client side configuration. (Boris Shkolnik via ddas)

  NEW FEATURES

    MAPREDUCE-1774. Large-scale Automated Framework (Sharad Agarwal, Sreekanth
    Ramakrishnan, Konstantin Boudnik, et al. via cos)

    MAPREDUCE-706. Support for FIFO pools in the fair scheduler.
    (Matei Zaharia)

    MAPREDUCE-546. Provide sample fair scheduler config file in conf/ and use
    it by default if no other config file is specified. (Matei Zaharia)

    MAPREDUCE-551. Preemption support in the Fair Scheduler. (Matei Zaharia)

    MAPREDUCE-567. Add a new example MR that always fails. (Philip Zeyliger
    via tomwhite)

    MAPREDUCE-211. Provides ability to run a health check script on the
    tasktracker nodes and blacklist nodes if they are unhealthy.
    (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-637. Add an example, distbbp, which able to compute the n th bit
    of Pi for some large n.  (szetszwo)

    MAPREDUCE-532. Provide a way to limit the number of used slots 
    per queue in the capacity scheduler.
    (Rahul Kumar Singh via yhemanth)

    MAPREDUCE-467. Provide ability to collect statistics about total tasks 
    and succeeded tasks in different time windows. (sharad)

    MAPREDUCE-740. Log a job-summary at the end of a job, while allowing it
    to be configured to use a custom appender if desired. (acmurthy)

    MAPREDUCE-814. Provide a way to configure completed job history files 
    to be on HDFS. (sharad)

    MAPREDUCE-800. MRUnit should support the new API. (Aaron Kimball via
    tomwhite)

    MAPREDUCE-798. MRUnit should be able to test a succession of MapReduce
    passes. (Aaron Kimball via tomwhite)

    MAPREDUCE-768. Provide an option to dump jobtracker configuration in JSON
    format to standard output. (V.V.Chaitanya Krishna via yhemanth)

    MAPREDUCE-824. Add support for a hierarchy of queues in the capacity 
    scheduler. (Rahul Kumar Singh via yhemanth)

    MAPREDUCE-751. Add Rumen, a tool for extracting statistics from job tracker
    logs and generating job traces for simulation and analysis. (Dick King via
    cdouglas)

    MAPREDUCE-830. Add support for splittable compression to TextInputFormats.
    (Abdul Qadeer via cdouglas)

    MAPREDUCE-861. Add support for hierarchical queues in the Map/Reduce
    framework. (Rahul Kumar Singh via yhemanth)

    MAPREDUCE-776. Add Gridmix, a benchmark processing Rumen traces to simulate
    a measured mix of jobs on a cluster. (cdouglas)

    MAPREDUCE-862. Enhance JobTracker UI to display hierarchical queues.
    (V.V.Chaitanya Krishna via yhemanth)

    MAPREDUCE-777. Brand new apis to track and query jobs as a
    replacement for JobClient. (Amareshwari Sriramadasu via acmurthy)

    MAPREDUCE-775. Add native and streaming support for Vertica as an input
    or output format taking advantage of parallel read and write properties of 
    the DBMS. (Omer Trajman via ddas)

    MAPREDUCE-679. XML-based metrics as JSP servlet for JobTracker.
    (Aaron Kimball via tomwhite)

    MAPREDUCE-980. Modify JobHistory to use Avro for serialization. (cutting)

    MAPREDUCE-728. Add Mumak, a Hadoop map/reduce simulator. (Arun C Murthy,
    Tamas Sarlos, Anirban Dasgupta, Guanying Wang, and Hong Tang via cdouglas)

    MAPREDUCE-1383. Automates fetching of delegation tokens in File*Formats
    Distributed Cache and Distcp. Also, provides a config 
    mapreduce.job.hdfs-servers that the jobs can populate with a comma
    separated list of namenodes. The job client automatically fetches
    delegation tokens from those namenodes. (Boris Shkolnik via ddas)    

    MAPREDUCE-698. Per-pool task limits for the fair scheduler.
    (Kevin Peterson via matei)

    MAPREDUCE-1026. Does mutual authentication of the shuffle
    transfers using a shared JobTracker generated key.
    (Boris Shkolnik via ddas)

    MAPREDUCE-744. Introduces the notion of a public distributed cache.
    (Devaraj Das)

    MAPREDUCE-1338. Introduces the notion of token cache using which
    tokens and secrets can be sent by the Job client to the JobTracker.
    (Boris Shkolnik via ddas)

    HDFS-503. This patch implements an optional layer over HDFS that 
    implements offline erasure-coding.  It can be used to reduce the 
    total storage requirements of HDFS.  (dhruba)

    MAPREDUCE-1432. Adds hooks in the jobtracker and tasktracker 
    for loading the tokens in the user's ugi. This is required
    for the copying of files from the hdfs. (ddas)

    MAPREDUCE-1335. Adds SASL Kerberos/Digest authentication in MapReduce.
    (Kan Zhang via ddas)

    MAPREDUCE-1464. Makes a compatible change in JobTokenIdentifier to
    account for HADOOP-6510. (Jitendra Nath Pandey via ddas)

    MAPREDUCE-1433. Add a delegation token for MapReduce. (omalley)

    MAPREDUCE-1307. Introduces the Job level ACLs feature. 
    (Vinod Kumar Vavilapalli via ddas)

    MAPREDUCE-1430. JobTracker automatically renews delegation tokens for jobs.
    (Boris Shkolnik via ddas)

    MAPREDUCE-1455. Introduces job-level authorization for mapreduce servlets.
    (Ravi Gummadi via vinodkv)

  IMPROVEMENTS

    MAPREDUCE-463. Makes job setup and cleanup tasks as optional.
    (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-502. Allow jobtracker to be configured with zero completed jobs
    in memory. (Amar Kamat via sharad)

    MAPREDUCE-416. Moves the history file to a "done" folder whenever a job 
    completes. (Amar Kamat via ddas)

    MAPREDUCE-646. Increase srcfilelist replication number in dictcp job.
    (Ravi Gummadi via szetszwo)
   
    HADOOP-6106. Updated hadoop-core and test jars from hudson trunk 
    build #12. (Giridharan Kesavan)

    MAPREDUCE-642. A option to distcp that allows preserving the full
    source path of a file in the specified destination directory.
    (Rodrigo Schmidt via dhruba)

    MAPREDUCE-686. Move TestSpeculativeExecution.Fake* into a separate class
    so that it can be used by other tests. (Jothi Padmanabhan via sharad)

    MAPREDUCE-625. Modify TestTaskLimits to improve execution time.
    (Jothi Padmanabhan via sharad)

    MAPREDUCE-465. Deprecate o.a.h.mapred.lib.MultithreadedMapRunner and add
    test for o.a.h.mapreduce.lib.MultithreadedMapper.
    (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-701. Improves the runtime of the TestRackAwareTaskPlacement
    by making it a unit test.  (Jothi Padmanabhan via ddas)

    MAPREDUCE-371. Change KeyFieldBasedComparator and KeyFieldBasedPartitioner
    to use new api. (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-623. Resolve javac warnings in mapreduce. (Jothi Padmanabhan
    via sharad)

    MAPREDUCE-655. Change KeyValueLineRecordReader and KeyValueTextInputFormat
    to use new mapreduce api. (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-632. Merge TestCustomOutputCommitter with 
    TestCommandLineJobSubmission. (Jothi Padmanabhan via sharad)

    MAPREDUCE-627. Improves execution time of TestTrackerBlacklistAcrossJobs.
    (Jothi Padmanabhan via ddas)

    MAPREDUCE-630. Improves execution time of TestKillCompletedJob.
    (Jothi Padmanabhan via ddas)

    MAPREDUCE-626. Improves the execution time of TestLostTracker.
    (Jothi Padmanabhan via ddas)

    MAPREDUCE-353. Makes the shuffle read and connection timeouts
    configurable. (Ravi Gummadi via ddas)

    MAPREDUCE-739. Allow relative paths to be created in archives. (Mahadev
    Konar via cdouglas)

    MAPREDUCE-772. Merge HADOOP-4010 changes to LineRecordReader into mapreduce
    package. (Abdul Qadeer via cdouglas)

    MAPREDUCE-785. Separate sub-test of TestReduceFetch to be included in
    MR-670. (Jothi Padmanabhan via cdouglas)

    MAPREDUCE-784. Modify TestUserDefinedCounters to use LocalJobRunner 
    instead of MiniMR. (Jothi Padmanabhan via sharad)

    HADOOP-6160. Fix releaseaudit target to run on specific directories. 
    (gkesavan)

    MAPREDUCE-782. Use PureJavaCrc32 in SpillRecord.  (Todd Lipcon via
    szetszwo)

    MAPREDUCE-369. Change org.apache.hadoop.mapred.lib.MultipleInputs to 
    use new api. (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-373. Change org.apache.hadoop.mapred.lib.FieldSelectionMapReduce
    to use new api. (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-628. Improves the execution time of TestJobInProgress.
    (Jothi Padmanabhan via ddas)

    MAPREDUCE-793. Creates a new test that consolidates a few tests to
    include in the commit-test list. (Jothi Padmanabhan via ddas)
    
    MAPREDUCE-797. Adds combiner support to MRUnit MapReduceDriver.
    (Aaron Kimball via johan)    

    MAPREDUCE-656. Change org.apache.hadoop.mapred.SequenceFile* classes
    to use new mapreduce api. (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-670. Creates ant target for 10 mins patch test build.
    (Jothi Padmanabhan via gkesavan)

    MAPREDUCE-375. Change org.apache.hadoop.mapred.lib.NLineInputFormat 
    and org.apache.hadoop.mapred.MapFileOutputFormat to use new api.
    (Amareshwari Sriramadasu via ddas)

    MAPREDUCE-779. Added node health failure counts into 
    JobTrackerStatistics. (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-842. Setup secure permissions for localized job files,
    intermediate outputs and log files on tasktrackers.
    (Vinod Kumar Vavilapalli via yhemanth)
    
    MAPREDUCE-478. Allow map and reduce jvm parameters, environment variables
    and ulimit to be set separately.
    Configuration changes:
      add mapred.map.child.java.opts
      add mapred.reduce.child.java.opts
      add mapred.map.child.env
      add mapred.reduce.child.ulimit
      add mapred.map.child.env
      add mapred.reduce.child.ulimit
      deprecated mapred.child.java.opts
      deprecated mapred.child.env
      deprecated mapred.child.ulimit
    (acmurthy)

    MAPREDUCE-767. Remove the dependence on the CLI 2.0 snapshot.
    (Amar Kamat via omalley)

    MAPREDUCE-712. Minor efficiency tweaks to RandomTextWriter. (cdouglas)

    MAPREDUCE-870. Remove the job retire thread and the associated 
    config parameters. (sharad)

    MAPREDUCE-874. Rename the PiEstimator example to QuasiMonteCarlo.
    (szetszwo)

    MAPREDUCE-336. Allow logging level of map/reduce tasks to be configurable. 
    Configuration changes:
      add mapred.map.child.log.level 
      add mapred.reduce.child.log.level 
    (acmurthy)

    MAPREDUCE-355. Update mapred.join package to use the new API. (Amareshwari
    Sriramadasu via cdouglas)

    HADOOP-6184. Updated hadoop common and test jars to get the new API
    in Configuration for dumping in JSON format from Hudson trunk build #68.
    (yhemanth)

    MAPREDUCE-476. Extend DistributedCache to work locally (LocalJobRunner).
    (Philip Zeyliger via tomwhite)

    MAPREDUCE-825. JobClient completion poll interval of 5s causes slow tests
    in local mode. (Aaron Kimball via tomwhite)

    MAPREDUCE-910. Support counters in MRUnit. (Aaron Kimball via cdouglas)

    MAPREDUCE-788. Update gridmix2 to use the new API (Amareshwari Sriramadasu
    via cdouglas)

    MAPREDUCE-875. Make DBRecordReader execute queries lazily. (Aaron Kimball 
    via enis)

    MAPREDUCE-318. Modularizes the shuffle code. (Jothi Padmanabhan and 
    Arun Murthy via ddas)

    MAPREDUCE-936. Allow a load difference for fairshare scheduler.
    (Zheng Shao via dhruba)

    MAPREDUCE-370. Update MultipleOutputs to use the API, merge funcitonality
    of MultipleOutputFormat. (Amareshwari Sriramadasu via cdouglas)

    MAPREDUCE-898. Changes DistributedCache to use the new API.
    (Amareshwari Sriramadasu via ddas)

    MAPREDUCE-144. Includes dump of the process tree in task diagnostics when 
    a task is killed due to exceeding memory limits.
    (Vinod Kumar Vavilapalli via yhemanth)

    MAPREDUCE-945. Modifies MRBench and TestMapRed to use ToolRunner so that
    options such as queue name can be passed via command line.
    (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-963. Deprecate o.a.h.mapred.FileAlreadyExistsException and
    replace it with o.a.h.fs.FileAlreadyExistsException.  (Boris Shkolnik
    via szetszwo)

    MAPREDUCE-960. Remove an unnecessary intermediate copy and obsolete API
    from KeyValueLineRecordReader. (cdouglas)

    MAPREDUCE-930. Modify Rumen to resolve paths in the canonical way, rather
    than defaulting to the local filesystem. (cdouglas)

    MAPREDUCE-944. Extend the LoadManager API of the fair-share scheduler
    to support regulating tasks for a job based on resources currently in use
    by that job. (dhruba)

    MAPREDUCE-973. Move FailJob and SleepJob from examples to test. (cdouglas 
    via omalley)

    MAPREDUCE-966. Modify Rumen to clean up interfaces and simplify integration
    with other tools. (Hong Tang via cdouglas)

    MAPREDUCE-856. Setup secure permissions for distributed cache files.
    (Vinod Kumar Vavilapalli via yhemanth)

    MAPREDUCE-885. More efficient SQL queries for DBInputFormat. (Aaron Kimball 
    via enis)

    MAPREDUCE-284. Enables ipc.client.tcpnodelay in Tasktracker's Child.
    (Ravi Gummadi via sharad)

    MAPREDUCE-916. Split the documentation to match the project split.
    (Corinne Chandel via omalley)

    MAPREDUCE-649. Validate a copy by comparing the source and destination
    checksums in distcp. Also adds an intra-task retry mechanism for errors
    detected during the copy. (Ravi Gummadi via cdouglas)

    MAPREDUCE-654. Add a -dryrun option to distcp printing a summary of the
    file data to be copied, without actually performing the copy. (Ravi Gummadi
    via cdouglas)

    MAPREDUCE-664. Display the number of files deleted by distcp when the
    -delete option is specified. (Ravi Gummadi via cdouglas)

    MAPREDUCE-781. Let the name of distcp jobs be configurable. (Venkatesh S
    via cdouglas)

    MAPREDUCE-975. Add an API in job client to get the history file url for 
    a given job id. (sharad)

    MAPREDUCE-905. Add Eclipse launch tasks for MapReduce. (Philip Zeyliger
    via tomwhite)

    MAPREDUCE-277. Makes job history counters available on the job history
    viewers. (Jothi Padmanabhan via ddas)

    MAPREDUCE-893. Provides an ability to refresh queue configuration
    without restarting the JobTracker.
    (Vinod Kumar Vavilapalli and Rahul Kumar Singh via yhemanth)

    MAPREDUCE-1011. Add build.properties to svn and git ignore. (omalley)

    MAPREDUCE-954. Change Map-Reduce context objects to be interfaces.
    (acmurthy) 

    MAPREDUCE-639. Change Terasort example to reflect the 2009 updates. 
    (omalley)

    MAPREDUCE-1063. Document gridmix benchmark. (cdouglas)

    MAPREDUCE-931. Use built-in interpolation classes for making up task
    runtimes in Rumen. (Dick King via cdouglas)

    MAPREDUCE-1012. Mark Context interfaces as public evolving. (Tom White via
    cdouglas)

    MAPREDUCE-971. Document use of distcp when copying to s3, managing timeouts
    in particular. (Aaron Kimball via cdouglas)

    HDFS-663. DFSIO for append. (shv)

    HDFS-641. Move all of the components that depend on map/reduce to 
    map/reduce. (omalley)

    HADOOP-5107. Use Maven ant tasks to publish artifacts. (Giridharan Kesavan
    via omalley)

    MAPREDUCE-1229. Allow customization of job submission policy in Mumak.
    (Hong Tang via cdouglas)

    MAPREDUCE-1317. Reduce the memory footprint of Rumen objects by interning
    host Strings. (Hong Tang via cdouglas)

    MAPREDUCE-1097. Add support for Vertica 3.5 to its contrib module. (Omer
    Trajman via cdouglas)

    MAPREDUCE-1627. HadoopArchives should not uses a method in DistCp.
    (szetszwo)

    MAPREDUCE-1198. Alternatively schedule different types of tasks in
    fair share scheduler. (Scott Chen via matei)

    MAPREDUCE-707. Provide a jobconf property for explicitly assigning a job to 
    a pool in the Fair Scheduler. (Alan Heirich via matei)

    MAPREDUCE-947. Added commitJob and abortJob apis to OutputCommitter.
    Enhanced FileOutputCommitter to create a _SUCCESS file for successful
    jobs. (Amar Kamat & Jothi Padmanabhan via acmurthy) 

    MAPREDUCE-1103. Added more metrics to Jobtracker. (sharad) 

    MAPREDUCE-1048. Add occupied/reserved slot usage summary on jobtracker UI.
    (Amareshwari Sriramadasu and Hemanth Yamijala via sharad)

    MAPREDUCE-1090. Modified log statement in TaskMemoryManagerThread to
    include task attempt id. (yhemanth)

    MAPREDUCE-1189. Reduce ivy console output to ovservable level (cos)

    MAPREDUCE-1167. ProcfsBasedProcessTree collects rss memory information.
    (Scott Chen via dhruba)

    MAPREDUCE-1231. Added a new DistCp option, -skipcrccheck, so that the CRC
    check during setup can be skipped.  (Jothi Padmanabhan via szetszwo)

    MAPREDUCE-1190. Add package documentation for BBP example.
    (Tsz Wo (Nicholas) Sze via cdouglas)

    MAPREDUCE-1119. When tasks fail to report status, show tasks's stack dump
    before killing. (Aaron Kimball via tomwhite)

    MAPREDUCE-1185. Redirect running job url to history url if job is already 
    retired. (Amareshwari Sriramadasu and Sharad Agarwal via sharad)

    MAPREDUCE-1050. Introduce a mock object testing framework. (tomwhite)

    MAPREDUCE-1084. Implementing aspects development and fault injeciton
    framework for MapReduce. (Sreekanth Ramakrishnan via cos)

    MAPREDUCE-1209. Move common specific part of the test TestReflectionUtils
    out of mapred into common. (Todd Lipcon via tomwhite)

    MAPREDUCE-967. TaskTracker does not need to fully unjar job jars.
    (Todd Lipcon via tomwhite)

    MAPREDUCE-1083. Changes in MapReduce so that group information of users 
    can be refreshed in the JobTracker via command line. 
    (Boris Shkolnik via ddas)

    MAPREDUCE-181. Changes the job submission process to be secure.
    (Devaraj Das)

    MAPREDUCE-1250. Refactors the JobToken to use Common's Token interface.
    (Kan Zhang via ddas)

    MAPREDUCE-896. Enhance tasktracker to cleanup files that might have
    been created by user tasks with non-writable permissions.
    (Ravi Gummadi via yhemanth)

    MAPREDUCE-372. Change org.apache.hadoop.mapred.lib.ChainMapper/Reducer 
    to use new mapreduce api. (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-1295. Add a tool in Rumen for folding and manipulating job
    traces. (Dick King via cdouglas)

    MAPREDUCE-1302. TrackerDistributedCacheManager deletes file
    asynchronously, thus reducing task initialization delays.
    (Zheng Shao via dhruba)

    MAPREDUCE-1218. TaskTrackers send cpu and memory usage of
    node to JobTracker. (Scott Chen via dhruba)

    MAPREDUCE-847. Fix Releaseaudit warning count to zero
    (Giridharan Kesavan)

    MAPREDUCE-1337. Use generics in StreamJob to improve readability of that
    class. (Kay Kay via cdouglas)

    MAPREDUCE-361. Port terasort example to the new mapreduce API. (Amareshwari
    Sriramadasu via cdouglas)

    MAPREDUCE-1367. LocalJobRunner should support parallel mapper execution.
    (Aaron Kimball via tomwhite)

    MAPREDUCE-64. Eliminate io.sort.record.percent from MapTask configuration.
    (cdouglas)

    MAPREDUCE-1440. Replace the long user name in MapReduce with the local
    name. (omalley)

    MAPREDUCE-1470. Move delegation tokens from HDFS to Common so that 
    MapReduce can use them too. (omalley)

    MAPREDUCE-1425. Reduce memory usage by archive. (mahadev via szetszwo)

    MAPREDUCE-1441. Trim whitespace from directory lists pulled from the
    configuration. (Todd Lipcon via cdouglas)

    MAPREDUCE-1309. Refactor Rumen trace generator to improve code structure
    and add extensible support for log formats. (Dick King via cdouglas)

    MAPREDUCE-1503. Delegation token renewing and cancelling should provide
    meaningful exceptions when there are failures instead of returning 
    false. (omalley)

    HADOOP-6579. Upgrade commons-codec library to 1.4. (omalley)

    MAPREDUCE-1423. Improve performance of CombineFileInputFormat when multiple
    pools are configured. (Dhruba Borthakur via zshao)

    MAPREDUCE-1454. Quote user supplied strings in Tracker servlets. (cdouglas)

    MAPREDUCE-1408. Add customizable job submission policies to Gridmix. (Rahul
    Singh via cdouglas)

    MAPREDUCE-1527. Better warning logged when mapred.queue.names is
    overshadowed by mapred-queues.xml. (Hong Tang via acmurthy)

    MAPREDUCE-1403. Save the size and number of distributed cache artifacts in
    the configuration. (Arun Murthy via cdouglas)

    MAPREDUCE-1482. Truncate state string and diagnostic information in
    TaskStatus. (Amar Kamat via szetszwo)

    MAPREDUCE-1593.  [Rumen] Improvements to random seed generation (tamas via
    mahadev)

    MAPREDUCE-1460. Oracle support in DataDrivenDBInputFormat.
    (Aaron Kimball via tomwhite)

    MAPREDUCE-1569. Pass configuration through mocked contexts in MRUnit.
    (Chris White via cdouglas)

    MAPREDUCE-1590. Move HarFileSystem from Hadoop Common to Mapreduce tools.
    (mahadev)

    MAPREDUCE-1629. Get rid of fakeBlockLocations() on HarFileSystem, since
    it's not used (mahadev)

    MAPREDUCE-1489. DataDrivenDBInputFormat should not query the database
    when generating only one split. (Aaron Kimball via tomwhite)

    MAPREDUCE-1514.  Add documentation on replication, permissions, new options,
    limitations and internals of har.  (mahadev via szetszwo)

    MAPREDUCE-1428.  Make block size and the size of archive created files
    configurable.  (mahadev via szetszwo)

    MAPREDUCE-1656. JobStory should provide queue info. (hong via mahadev)

    MAPREDUCE-1466. Record number of files processed in FileInputFormat in the
    Configuration for offline analysis. (Luke Lu and Arun Murthy via cdouglas)

    MAPREDUCE-1538. TrackerDistributedCacheManager manages the
    number of files. (Scott Chen via dhruba)

    MAPREDUCE-1673. Scripts to start and stop RaidNode.
    (Rodrigo Schmidt via dhruba)

    MAPREDUCE-1659. RaidNode writes temp files on configured tmp directory and
    add random numbers to their names to avoid conflicts
    (Rodrigo Schmidt via dhruba)

    MAPREDUCE-1221. Allow admins to control physical memory limits per-task
    and per-node. (Scott Chen via acmurthy) 

    MAPREDUCE-1065. Update mapred tutorial to use the new API. (Aaron Kimball
    via cdouglas)

    MAPREDUCE-1304. Add a task counter tracking time spent in GC. (Aaron
    Kimball via cdouglas)

    MAPREDUCE-1570. Add grouping comparators to MRUnit. (Chris White via
    cdouglas)

    MAPREDUCE-1650. Exclude Private elements from generated MapReduce
    Javadoc. (tomwhite)

    MAPREDUCE-1625. Improve grouping of packages in Javadoc. (tomwhite)

    MAPREDUCE-1417. Forrest documentation should be updated to reflect 
    the changes in MAPREDUCE-744. (Ravi Gummadi via vinodkv)

    MAPREDUCE-1568. TrackerDistributedCacheManager should clean up cache
    in a background thread. (Scott Chen via zshao)

    MAPREDUCE-1749. Move configuration strings out of JobContext so that it
    can be made public stable. (omalley)

    MAPREDUCE-1623. Apply audience and stability notations to Hadoop
    Map-Reduce. (tomwhite via acmurthy) 

    MAPREDUCE-1751. Change MapReduce to depend on Hadoop 'common' artifacts
    instead of 'core'. (tomwhite)

    MAPREDUCE-1535.  Replace usage of FileStatus#isDir().  (Eli Collins via
    tomwhite)

    MAPREDUCE-1832. Allow file sizes less than 1MB in DFSIO benchmark. (shv)

    MAPREDUCE-1404.  Move Cluster-Setup and Single-Node-Setup Docs from
    MapReduce to Common.  (tomwhite)

    MAPREDUCE-1697. Document the behavior of -file option and deprecate it
    in favour of -files option in streaming. (Amareshwari Sriramadasu
    via vinodkv)

    MAPREDUCE-1033. Resolve location of scripts and configuration files after
    project split. (tomwhite)

    MAPREDUCE-1018. Document changes to the memory management and scheduling
    model. (Hemanth Yamijala via vinodkv)

    MAPREDUCE-1896. [Herriot] New property for multi user list. (Vinay Thota
    via cos)

    MAPREDUCE-1812. New properties for suspend and resume process. (Vinay
    Thota via cos)

  OPTIMIZATIONS

    MAPREDUCE-270. Fix the tasktracker to optionally send an out-of-band
    heartbeat on task-completion for better job-latency. (acmurthy) 
    Configuration changes:
      add mapreduce.tasktracker.outofband.heartbeat 

    MAPREDUCE-1186. Modified code in distributed cache to set permissions
    only on required set of localized paths.
    (Amareshwari Sriramadasu via yhemanth)

    MAPREDUCE-1501. FileInputFormat supports multi-level, recursive 
    directory listing.  (Zheng Shao via dhruba)

    MAPREDUCE-1556. upgrade to Avro 1.3.0. (cutting via tomwhite)

    MAPREDUCE-1613. Install/deploy source jars to Maven repo 
    (Patrick Angeles via ddas)

    MAPREDUCE-1610. Forrest documentation should be updated to reflect
    the changes in MAPREDUCE-856. (Ravi Gummadi via vinodkv)

    MAPREDUCE-1853. Adds caching for TaskAttemptContext in MultipleOutputs.
    (Torsten Curdt via amareshwari)

  BUG FIXES

    MAPREDUCE-878. Rename fair scheduler design doc to 
    fair-scheduler-design-doc.tex and add Apache license header (matei)

    HADOOP-4687. MapReduce is split from Hadoop Core. It is a subproject under 
    Hadoop (Owen O'Malley)

    HADOOP-6096. Fix Eclipse project and classpath files following project
    split. (tomwhite)

    MAPREDUCE-419. Reconcile mapred.userlog.limit.kb defaults in configuration
    and code. (Philip Zeyliger via cdouglas)

    MAPREDUCE-2. Fixes a bug in KeyFieldBasedPartitioner in handling empty
    keys. (Amar Kamat via sharad)

    MAPREDUCE-130. Delete the jobconf copy from the log directory of the 
    JobTracker when the job is retired. (Amar Kamat via sharad)

    MAPREDUCE-657. Fix hardcoded filesystem problem in CompletedJobStatusStore.
    (Amar Kamat via sharad)

    MAPREDUCE-179. Update progress in new RecordReaders. (cdouglas)

    MAPREDUCE-658. Replace NPE in distcp with a meaningful error message when
    the source path does not exist. (Ravi Gummadi via cdouglas)

    MAPREDUCE-671. Update ignore list to include untracked, generated
    build artifacts and config files. (cdouglas)

    MAPREDUCE-433. Use more reliable counters in TestReduceFetch. (cdouglas)

    MAPREDUCE-124. Fix a bug in failure handling of abort task of 
    OutputCommiter. (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-694. Fix to add jsp-api jars to capacity-scheduler classpath.
    (Giridharan Kesavan)
    
    MAPREDUCE-702. Fix eclipse-plugin jar target (Giridharan Kesavan) 

    MAPREDUCE-522. Replace TestQueueCapacities with simpler test case to
    test integration between capacity scheduler and MR framework.
    (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-683. Fixes an initialization problem in the JobHistory. 
    The initialization of JobHistoryFilesManager is now done in the 
    JobHistory.init call. (Amar Kamat via ddas)

    MAPREDUCE-708. Fixes a bug to allow updating the reason for
    blacklisting a node on the JobTracker UI.
    (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-709. Fixes message displayed for a blacklisted node where
    the reason for blacklisting is due to the health check script
    timing out. (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-676. Existing diagnostic rules fail for MAP ONLY jobs.
    (Suhas Gogate via tomwhite)

    MAPREDUCE-722. Fixes a bug with tasktracker reservations for
    high memory jobs in capacity scheduler.
    (Vinod Kumar Vavilapalli via yhemanth)

    HADOOP-6090. Updates gridmix script to use new mapreduce api output 
    format. (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-732. Removed spurious log statements in the node
    blacklisting logic. (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-734. Fix a ConcurrentModificationException in unreserving
    unused reservations for a job when it completes.
    (Arun Murthy and Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-733. Fix a RuntimeException while unreserving trackers
    that are blacklisted for a job.
    (Arun Murthy and Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-677. Fix timeout in TestNodeRefresh. (Amar Kamat via 
    sharad)

    MAPREDUCE-153. Fix timeout in TestJobInProgressListener. (Amar 
    Kamat via sharad)

    MAPREDUCE-742. Fix output messages and java comments in the Pi related
    examples.  (szetszwo)

    MAPREDUCE-565. Fix partitioner to work with new API. (Owen O'Malley via
    cdouglas)
    
    MAPREDUCE-680. Fix so MRUnit can handle reuse of Writable objects.
    (Aaron Kimball via johan)

    MAPREDUCE-18. Puts some checks for cross checking whether a reduce
    task gets the correct shuffle data. (Ravi Gummadi via ddas)

    MAPREDUCE-771. Fix scheduling of setup and cleanup tasks to use
    free slots instead of tasks for scheduling. (yhemanth)

    MAPREDUCE-717. Fixes some corner case issues in speculative 
    execution heuristics. (Devaraj Das)

    MAPREDUCE-716. Make DBInputFormat work with Oracle. (Aaron Kimball
    via tomwhite)

    MAPREDUCE-735. Fixes a problem in the KeyFieldHelper to do with 
    the end index for some inputs (Amar Kamat via ddas)

    MAPREDUCE-682. Removes reservations on tasktrackers which are
    blacklisted. (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-743. Fixes a problem to do with progress reporting
    in the map phase. (Ravi Gummadi via ddas)

    MAPREDUCE-765. Eliminate the deprecated warnings introduced by H-5438.
    (He Yongqiang via szetszwo)

    MAPREDUCE-383. Fix a bug in Pipes combiner due to bytes count not 
    getting reset after the spill. (Christian Kunz via sharad)

    MAPREDUCE-809. Fix job-summary logs to correctly record status of FAILED
    and KILLED jobs.  (acmurthy)

    MAPREDUCE-792. Fix unchecked warnings in DBInputFormat.  (Aaron Kimball
    via szetszwo)

    MAPREDUCE-760. Fix a timing issue in TestNodeRefresh. (Amar Kamat via 
    sharad)

    MAPREDUCE-40. Keep memory management backwards compatible for job
    configuration parameters and limits. (Rahul Kumar Singh via yhemanth)

    MAPREDUCE-587. Fixes a OOM issue in TestStreamingExitStatus.
    (Amar Kamat via ddas) 

    MAPREDUCE-408. Fixes an assertion problem in TestKillSubProcesses
    (Ravi Gummadi via ddas)
     
    MAPREDUCE-659. Fix gridmix2 compilation. (Giridharan Kesavan)

    MAPREDUCE-796. Fixes a ClassCastException in an exception log in
    MultiThreadedMapRunner. (Amar Kamat via ddas)

    MAPREDUCE-808. Fixes a serialization problem in TypedBytes.
    (Klaas Bosteels via ddas)

    MAPREDUCE-845. Fix a findbugs heap size problem in build.xml and add
    a new property findbugs.heap.size.  (Lee Tucker via szetszwo)

    MAPREDUCE-838. Fixes a problem in the way commit of task outputs
    happens. The bug was that even if commit failed, the task would
    be declared as successful. (Amareshwari Sriramadasu via ddas)

    MAPREDUCE-813. Updates Streaming and M/R tutorial documents.
    (Corinne Chandel via ddas)

    MAPREDUCE-805. Fixes some deadlocks in the JobTracker due to the fact
    the JobTracker lock hierarchy wasn't maintained in some JobInProgress
    method calls. (Amar Kamat via ddas)
    
    MAPREDUCE-799. Fixes so all of the MRUnit self-tests run.
    (Aaron Kimball via johan)    

    MAPREDUCE-848. Fixes a problem to do with TestCapacityScheduler
    failing (Amar Kamat via ddas)

    MAPREDUCE-840. DBInputFormat leaves open transaction.
    (Aaron Kimball via tomwhite)

    MAPREDUCE-859. Adds Avro and its dependencies required by Hadoop 
    common. (Ravi Gummadi via sharad)
    
    MAPREDUCE-867. Fix ivy conf to look for avro jar from maven repo.	
    (Giridharan Kesavan)

    MAPREDUCE-877. Added avro as a dependency to contrib ivy settings.
    (Tsz Wo (Nicholas) Sze via yhemanth)

    MAPREDUCE-852. In build.xml, remove the Main-Class, which is incorrectly
    set in tools, and rename the target "tools-jar" to "tools".  (szetszwo)

    MAPREDUCE-773. Sends progress reports for compressed gzip inputs in maps.
    Fixes a native direct buffer leak in LineRecordReader classes.
    (Hong Tang and ddas)

    MAPREDUCE-832. Reduce number of warning messages printed when
    deprecated memory variables are used. (Rahul Kumar Singh via yhemanth)

    MAPREDUCE-745. Fixes a testcase problem to do with generation of JobTracker
    IDs. (Amar Kamat via ddas)

    MAPREDUCE-834. Enables memory management on tasktrackers when old
    memory management parameters are used in configuration.
    (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-818. Fixes Counters#getGroup API. (Amareshwari Sriramadasu 
    via sharad)

    MAPREDUCE-807. Handles the AccessControlException during the deletion of
    mapred.system.dir in the JobTracker. The JobTracker will bail out if it
    encounters such an exception. (Amar Kamat via ddas)

    MAPREDUCE-430. Fix a bug related to task getting stuck in case of 
    OOM error. (Amar Kamat via ddas)

    MAPREDUCE-871. Fix ownership of Job/Task local files to have correct 
    group ownership according to the egid of the tasktracker.
    (Vinod Kumar Vavilapalli via yhemanth) 

    MAPREDUCE-911. Fix a bug in TestTaskFail related to speculative 
    execution. (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-687. Fix an assertion in TestMiniMRMapRedDebugScript.
    (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-924. Fixes the TestPipes testcase to use Tool.
    (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-903. Add Avro jar to eclipse classpath.
    (Philip Zeyliger via tomwhite)

    MAPREDUCE-943. Removes a testcase in TestNodeRefresh that doesn't make 
    sense in the new Job recovery model. (Amar Kamat via ddas)

    MAPREDUCE-764. TypedBytesInput's readRaw() does not preserve custom type
    codes. (Klaas Bosteels via tomwhite)

    HADOOP-6243. Fixes a NullPointerException in handling deprecated keys.
    (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-968. NPE in distcp encountered when placing _logs directory on
    S3FileSystem. (Aaron Kimball via tomwhite)
 
    MAPREDUCE-826. harchive doesn't use ToolRunner / harchive returns 0 even
    if the job fails with exception (koji Noguchi via mahadev)

    MAPREDUCE-839. unit test TestMiniMRChildTask fails on mac os-x (hong tang
    via mahadev)

    MAPREDUCE-112. Add counters for reduce input, output records to the new API.
    (Jothi Padmanabhan via cdouglas)

    MAPREDUCE-648. Fix two distcp bugs: (1) it should not launch a job if all
    src paths are directories, and (2) it does not skip copying when updating
    a single file.  (Ravi Gummadi via szetszwo)

    MAPREDUCE-946. Fix a regression in LineRecordReader where the
    maxBytesToConsume parameter is not set correctly. (cdouglas)

    MAPREDUCE-977. Missing jackson jars from Eclipse template. (tomwhite)

    MAPREDUCE-988. Fix a packaging issue in the contrib modules. (Hong Tang via
    cdouglas)

    MAPREDUCE-971. distcp does not always remove distcp.tmp.dir. (Aaron Kimball
    via tomwhite)

    MAPREDUCE-995. Fix a bug in JobHistory where tasks completing after the job
    is closed cause a NPE. (Jothi Padmanabhan via cdouglas)

    MAPREDUCE-953. Fix QueueManager to dump queue configuration in JSON format.
    (V.V. Chaitanya Krishna via yhemanth)

    MAPREDUCE-645. Prevent distcp from running a job when the destination is a
    file, but the source is not. (Ravi Gummadi via cdouglas)

    MAPREDUCE-1002. Flushed writer in JobQueueClient so queue information is
    printed correctly. (V.V. Chaitanya Krishna via yhemanth)

    MAPREDUCE-1003. Fix compilation problem in eclipse plugin when
    eclipse.home is set. (Ravi Gummadi via yhemanth)

    MAPREDUCE-941. Vaidya script fails on Solaris. (Chad Metcalf
    via tomwhite)

    MAPREDUCE-912. Add and standardize Apache license headers. (Chad Metcalf
    via cdouglas)

    MAPREDUCE-1022. Fix compilation of vertica testcases. (Vinod Kumar 
    Vavilapalli via acmurthy)

    MAPREDUCE-1000. Handle corrupt history files in JobHistory.initDone().
    (Jothi Padmanabhan via sharad)

    MAPREDUCE-1028. Fixed number of slots occupied by cleanup tasks to one
    irrespective of slot size for the job.
    (Ravi Gummadi via yhemanth)

    MAPREDUCE-964. Fixed start and finish times of TaskStatus to be
    consistent, thereby fixing inconsistencies in metering tasks.
    (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-1076. Deprecate ClusterStatus and add javadoc in ClusterMetrics.
    (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-979. Fixed JobConf APIs related to memory parameters to return
    values of new configuration variables when deprecated variables are
    disabled. (Sreekanth Ramakrishnan via yhemanth)
   
    MAPREDUCE-1030. Modified scheduling algorithm to return a map and reduce
    task per heartbeat in the capacity scheduler.
    (Rahul Kumar Singh via yhemanth)

    MAPREDUCE-1071. Use DataInputStream rather than FSDataInputStream in the
    JobHistory EventReader. (Hong Tang via cdouglas)

    MAPREDUCE-986. Fix Rumen to work with truncated task lines. (Dick King via
    cdouglas)

    MAPREDUCE-1029. Fix failing TestCopyFiles by restoring the unzipping of
    HDFS webapps from the hdfs jar. (Aaron Kimball and Jothi Padmanabhan via
    cdouglas)

    MAPREDUCE-769. Make findbugs and javac warnings to zero.
    (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-1104. Initialize RecoveryManager in JobTracker cstr called by
    Mumak. (Hong Tang via cdouglas)

    MAPREDUCE-1061. Add unit test validating byte specifications for gridmix
    jobs. (cdouglas)

    MAPREDUCE-1077. Fix Rumen so that truncated tasks do not mark the job as
    successful. (Dick King via cdouglas)

    MAPREDUCE-1041. Make TaskInProgress::taskStatuses map package-private.
    (Jothi Padmanabhan via cdouglas)

    MAPREDUCE-1070. Prevent a deadlock in the fair scheduler servlet.
    (Todd Lipcon via cdouglas)

    MAPREDUCE-1086. Setup Hadoop logging environment for tasks to point to
    task related parameters. (Ravi Gummadi via yhemanth)

    MAPREDUCE-1105. Remove max limit configuration in capacity scheduler in
    favor of max capacity percentage thus allowing the limit to go over
    queue capacity. (Rahul Kumar Singh via yhemanth)

    MAPREDUCE-1016.  Make the job history log format JSON.  (cutting)

    MAPREDUCE-1038. Weave Mumak aspects only if related files have changed.
    (Aaron Kimball via cdouglas)

    MAPREDUCE-1163. Remove unused, hard-coded paths from libhdfs. (Allen
    Wittenauer via cdouglas)

    MAPREDUCE-962. Fix a NullPointerException while killing task process 
    trees. (Ravi Gummadi via yhemanth)

    MAPREDUCE-1177. Correct setup/cleanup inversion in
    JobTracker::getTaskReports. (Vinod Kumar Vavilapalli via cdouglas)

    MAPREDUCE-1178. Fix ClassCastException in MultipleInputs by adding 
    a DelegatingRecordReader. (Amareshwari Sriramadasu and Jay Booth 
    via sharad)

    MAPREDUCE-1068. Fix streaming job to show proper message if file is 
    is not present. (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-1147. Add map output counters to new API. (Amar Kamat via
    cdouglas)

    MAPREDUCE-915. The debug scripts are run as the job user. (ddas)

    MAPREDUCE-1007. Fix NPE in CapacityTaskScheduler.getJobs(). 
    (V.V.Chaitanya Krishna via sharad)

    MAPREDUCE-28. Refactor TestQueueManager and fix default ACLs.
    (V.V.Chaitanya Krishna and Rahul K Singh via sharad)

    MAPREDUCE-1182. Fix overflow in reduce causing allocations to exceed the
    configured threshold. (cdouglas)

    MAPREDUCE-1239. Fix contrib components build dependencies. 
    (Giridharan Kesavan and omalley) 

    MAPREDUCE-787. Fix JobSubmitter to honor user given symlink path.
    (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-1249. Update config default value for socket read timeout to
    match code default. (Amareshwari Sriramadasu via cdouglas)

    MAPREDUCE-1161. Remove ineffective synchronization in NotificationTestCase.
    (Owen O'Malley via cdouglas)

    MAPREDUCE-1244. Fix eclipse-plugin's build dependencies. (gkesavan)

    MAPREDUCE-1075. Fix JobTracker to not throw an NPE for a non-existent
    queue. (V.V.Chaitanya Krishna via yhemanth)

    MAPREDUCE-754. Fix NPE in expiry thread when a TT is lost. (Amar Kamat 
    via sharad)

    MAPREDUCE-1074. Document Reducer mark/reset functionality. (Jothi
    Padmanabhan via cdouglas)

    MAPREDUCE-1267. Fix typo in mapred-default.xml. (Todd Lipcon via cdouglas)

    MAPREDUCE-952. Remove inadvertently reintroduced Task.Counter enum. (Jothi
    Padmanabhan via cdouglas)

    MAPREDUCE-1230. Fix handling of null records in VerticaInputFormat. (Omer
    Trajman via cdouglas)

    MAPREDUCE-1171. Allow shuffle retries and read-error reporting to be
    configurable. (Amareshwari Sriramadasu via acmurthy)

    MAPREDUCE-879. Fix broken unit test TestTaskTrackerLocalization on MacOS.
    (Sreekanth Ramakrishnan via yhemanth)

    MAPREDUCE-1124. Fix imprecise byte counts in Gridmix. (cdouglas)

    MAPREDUCE-1222. Add an option to exclude numeric IP addresses in topologies
    processed by Mumak. (Hong Tang via cdouglas)

    MAPREDUCE-1284. Fix fts_open() call in task-controller that was failing
    LinuxTaskController unit tests. (Ravi Gummadi via yhemanth)

    MAPREDUCE-1143. Fix running task counters to be updated correctly
    when speculative attempts are running for a TIP.
    (Rahul Kumar Singh via yhemanth)

    MAPREDUCE-1241. Use a default queue configuration in JobTracker when
    mapred-queues.xml is unavailable. (Todd Lipcon via cdouglas)

    MAPREDUCE-1301. Fix set up of permission checking script used in 
    localization tests. (Amareshwari Sriramadasu via yhemanth)

    MAPREDUCE-1286. Remove quoting from client opts in TaskRunner. (Yuri
    Pradkin via cdouglas)

    MAPREDUCE-1059. Use distcp.bytes.per.map when adding sync markers in
    distcp. (Aaron Kimball via cdouglas)

    MAPREDUCE-1009. Update forrest documentation describing hierarchical
    queues. (Vinod Kumar Vavilapalli via yhemanth)

    MAPREDUCE-1342. Fixed deadlock in global blacklisting of tasktrackers.
    (Amareshwari Sriramadasu via acmurthy)

    MAPREDUCE-1316. Fixes a memory leak of TaskInProgress instances in
    the jobtracker. (Amar Kamat via yhemanth)

    MAPREDUCE-1359. TypedBytes TestIO doesn't mkdir its test dir first.
    (Anatoli Fomenko via cos)

    MAPREDUCE-1314. Correct errant mapreduce.x.mapreduce.x replacements from
    bulk change. (Amareshwari Sriramadasu via cdouglas)

    MAPREDUCE-1365. Restore accidentally renamed test in
    TestTaskTrackerBloacklisting. (Amareshwari Sriramadasu via cdouglas)

    MAPREDUCE-1406. Fix spelling of JobContext.MAP_COMBINE_MIN_SPILLS.
    (cdouglas)

    MAPREDUCE-1369. JUnit tests should never depend on anything in conf
    (Anatoli Fomenko via cos)

    MAPREDUCE-1412. Fix timer granularity issue causing failures in
    TestTaskTrackerBlacklisting. (cdouglas)

    MAPREDUCE-1448. Respect --config option in Mumak script. (Hong Tang via
    cdouglas)

    MAPREDUCE-1251. c++ utils doesn't compile. (Eli Collins via tomwhite)

    MAPREDUCE-1522. FileInputFormat may use the default FileSystem for the
    input path. (Tsz Wo (Nicholas), SZE via cdouglas)

    MAPREDUCE-1407. Update javadoc in mapreduce.{Mapper,Reducer} to match
    actual usage. (Benoit Sigoure via cdouglas)

    MAPREDUCE-1258. Fix fair scheduler event log not logging job info.
    (matei)

    MAPREDUCE-1089. Fix NPE in fair scheduler preemption when tasks are
    scheduled but not running. (Todd Lipcon via matei)

    MAPREDUCE-1014. Fix the libraries for common and hdfs. (omalley)

    MAPREDUCE-1111. JT Jetty UI not working if we run mumak.sh 
    off packaged distribution directory. (hong tang via mahadev)

    MAPREDUCE-1133. Eclipse .classpath template has outdated jar files and is
    missing some new ones. (cos)

    MAPREDUCE-1098. Fixed the distributed-cache to not do i/o while holding a
    global lock. (Amareshwari Sriramadasu via acmurthy)

    MAPREDUCE-1158. Fix JT running maps and running reduces metrics.
    (sharad)

    MAPREDUCE-1160. Reduce verbosity of log lines in some Map/Reduce classes
    to avoid filling up jobtracker logs on a busy cluster.
    (Ravi Gummadi and Hong Tang via yhemanth)

    MAPREDUCE-1153. Fix tasktracker metrics when trackers are decommissioned.
    (sharad)

    MAPREDUCE-1128. Fix MRUnit to prohibit iterating over values twice. (Aaron
    Kimball via cdouglas)

    MAPREDUCE-665. Move libhdfs to HDFS subproject. (Eli Collins via dhruba)

    MAPREDUCE-1196. Fix FileOutputCommitter to use the deprecated cleanupJob
    api correctly. (acmurthy)
   
    MAPREDUCE-1244. Fix eclipse-plugin's build dependencies. (gkesavan)

    MAPREDUCE-1140. Fix DistributedCache to not decrement reference counts for
    unreferenced files in error conditions.
    (Amareshwari Sriramadasu via yhemanth)

    MAPREDUCE-1245. Fix TestFairScheduler failures by instantiating lightweight
    Jobtracker. (sharad)

    MAPREDUCE-1260. Update Eclipse configuration to match changes to Ivy
    configuration. (Edwin Chan via cos)

    MAPREDUCE-1152. Distinguish between failed and killed tasks in
    JobTrackerInstrumentation. (Sharad Agarwal via cdouglas)

    MAPREDUCE-1285. In DistCp.deleteNonexisting(..), get class from the
    parameter instead of using FileStatus.class.  (Peter Romianowski via
    szetszwo)

    MAPREDUCE-1294. Build fails to pull latest hadoop-core-* artifacts (cos)

    MAPREDUCE-1213. TaskTrackers restart is faster because it deletes
    distributed cache directory asynchronously. (Zheng Shao via dhruba)

    MAPREDUCE-1265. The task attempt error log prints the name of the 
    tasktracker machine. (Scott Chen via dhruba)

    MAPREDUCE-1201. ProcfsBasedProcessTree collects CPU usage information.
    (Scott Chen via dhruba)

    MAPREDUCE-1326. fi tests don't use fi-site.xml (cos)

    MAPREDUCE-1165. Replace non-portable function name with C99 equivalent.
    (Allen Wittenauer via cdouglas)

    MAPREDUCE-1331. Fixes a typo in a testcase (Devaraj Das)

    MAPREDUCE-1293. AutoInputFormat doesn't work with non-default FileSystems.
    (Andrew Hitchcock via tomwhite)

    MAPREDUCE-1131. Using profilers other than hprof can cause JobClient to
    report job failure. (Aaron Kimball via tomwhite)

    MAPREDUCE-1155. Streaming tests swallow exceptions.
    (Todd Lipcon via tomwhite)

    MAPREDUCE-1212. Mapreduce contrib project ivy dependencies are not included
    in binary target. (Aaron Kimball via tomwhite)

    MAPREDUCE-1388. Move the HDFS RAID package from HDFS to MAPREDUCE.
    (Eli Collins via dhruba)

    MAPREDUCE-1322. Defines default value for staging directory to be user
    based fixing a failing streaming test.
    (Devaraj Das and Amareshwari Sriramadasu via yhemanth)

    MAPREDUCE-899. Modified LinuxTaskController to check that task-controller
    has right permissions and ownership before performing any actions.
    (Amareshwari Sriramadasu via yhemanth)

    MAPREDUCE-1443. DBInputFormat can leak connections.
    (Aaron Kimball via tomwhite)

    MAPREDUCE-1457. Fixes JobTracker to get the FileSystem object within 
    getStagingAreaDir within a privileged block. Fixes Child.java to use the
    appropriate UGIs while getting the TaskUmbilicalProtocol proxy and 
    while executing the task. (Jakob Homan via ddas)

    MAPREDUCE-1399. The archive command shows a null error message (nicholas
    via mahadev)

    MAPREDUCE-1305. Improve efficiency of distcp -delete. (Peter Romianowski
    via cdouglas)

    MAPREDUCE-1474. Update forrest documentation for Hadoop Archives. (Mahadev
    Konar via cdouglas)

    MAPREDUCE-1400. Use tr rather than sed to effect literal substitution in
    the build script. (Allen Wittenauer via cdouglas)

    MAPREDUCE-1358. Avoid false positives in OutputLogFilter. (Todd Lipcon via
    cdouglas)

    MAPREDUCE-1490. Fix a NullPointerException that could occur during 
    instantiation and initialization of the DistributedRaidFileSystem. 
    (Rodrigo Schmidt via dhruba)

    MAPREDUCE-1476. Fix the M/R framework to not call commit for special
    tasks like job setup/cleanup and task cleanup.
    (Amareshwari Sriramadasu via yhemanth)

    MAPREDUCE-1398. Fix TaskLauncher to stop waiting for slots on a TIP that
    is killed / failed.
    (Amareshwari Sriramadasu via yhemanth)

    MAPREDUCE-1491. The parity files created by the RAID are combined
    using Hadoop Archive Files (HAR).  (Rodrigo Schmidt via dhruba)

    MAPREDUCE-1378. URL encode link in jobhistory.jsp to avoid errors caused by
    unescaped characters. (E. Sammer via cdouglas)

    MAPREDUCE-1519. RaidNode fails to create new parity file 
    if an older version already exists. (Rodrigo Schmidt via dhruba)

    MAPREDUCE-1537. Fixes a compilation problem in a testcase after commit
    HDFS-984. (Jitendra Nath Pandey via ddas)

    MAPREDUCE-1537. The patch makes the job client call the getDelegationToken
    only when security is enabled. (Jitendra Nath Pandey via ddas)

    MAPREDUCE-1510. RAID should regenerate parity files if they get deleted.
    (Rodrigo Schmidt via dhruba)

    MAPREDUCE-1421. Fix the LinuxTaskController tests failing on trunk after
    the commit of MAPREDUCE-1385. (Amareshwari Sriramadasu via vinodkv)

    MAPREDUCE-1520. Fix TestMiniMRLocalFS failure caused by regression in
    getting user working dir. (Amareshwari Sriramadasu via cdouglas)

    MAPREDUCE-1512. RAID uses HarFileSystem directly instead of
    FileSystem.get (Rodrigo Schmidt via dhruba)

    MAPREDUCE-1435. Fix symlink handling in task work directory when
    cleaning up, essentially to avoid following links.
    (Ravi Gummadi via yhemanth)

    MAPREDUCE-1518. RaidNode does not run the deletion check on the
    directory that stores the parity files.  (Rodrigo Schmidt via dhruba)

    MAPREDUCE-1573. TestStreamingAsDifferentUser fails if run as tt_user.
    (Ravi Gummadi via vinodkv)

    MAPREDUCE-927. Cleanup of task-logs should happen in TaskTracker instead
    of the Child. (Amareshwari Sriramadasu via vinodkv)

    MAPREDUCE-1578. Decouple HadoopArchives vesrion from HarFileSystem version.
    (Rodrigo Schmidt via szetszwo)

    MAPREDUCE-1422. Fix cleanup of localized job directory to work if files
    with non-deletable permissions are created within it.
    (Amar Kamat via yhemanth)

    MAPREDUCE-1306. Randomize the arrival of heartbeat responses in Mumak.
    (Tamas Sarlos via cdouglas)

    MAPREDUCE-1579. archive: check and possibly replace the space charater
    in source paths.  (szetszwo)

    MAPREDUCE-1536. DataDrivenDBInputFormat does not split date columns correctly.
    (Aaron Kimball via enis)

    MAPREDUCE-890. After HADOOP-4491, the user who started mapred system is 
    not able to run job. (Ravi Gummadi via vinodkv)

    MAPREDUCE-1615. Fix compilation of TestSubmitJob. (cdouglas)

    MAPREDUCE-1508. Protect against NPE in TestMultipleLevelCaching. (Aaron
    Kimball via cdouglas)

    MAPREDUCE-1497. Suppress spurious findbugs warning about IndexCache
    synchronization. (Amareshwari Sriramadasu via cdouglas)

    MAPREDUCE-1420. Fix TestTTResourceReporting failure. (Scott Chen via
    cdouglas)

    MAPREDUCE-1480. Correctly initialize child RecordReaders in
    CombineFileInputFormat. (Aaron Kimball via cdouglas)

    MAPREDUCE-1348. Fix block forensics packaging. (Tom White via cdouglas)

    MAPREDUCE-1628. HarFileSystem shows incorrect replication numbers and
    permissions.  (szetszwo via mahadev)

    MAPREDUCE-1602. Fix the error message for the case that src does not
    exist.  (szetszwo)

    MAPREDUCE-1585. Create Hadoop Archives version 2 with filenames
    URL-encoded (rodrigo via mahadev)

    MAPREDUCE-1523. Sometimes rumen trace generator fails to extract the job
    finish time. (dick king via mahadev)

    MAPREDUCE-1635. ResourceEstimator does not work after MAPREDUCE-842.
    (Amareshwari Sriramadasu via vinodkv)

    MAPREDUCE-889. binary communication formats added to Streaming by
    HADOOP-1722 should be documented. (Klaas Bosteels via tomwhite)

    MAPREDUCE-1031. ant tar target doens't seem to compile tests in contrib
    projects. (Aaron Kimball via tomwhite)

    MAPREDUCE-1692. Removed unused testcase TestStreamedMerge.
    (Sreekanth Ramakrishnan and Amareshwari Sriramadasu via yhemanth)

    MAPREDUCE-1062. Fix ReliabilityTest to work with retired jobs. (Sreekanth
    Ramakrishnan via cdouglas)

    MAPREDUCE-1409. IOExceptions thrown from FIleOutputCommitter::abortTask
    should cause the task to fail. (Amareshwari Sriramadasu via cdouglas)

    MAPREDUCE-1695. Include capacity scheduler in findbugs and javadoc-dev
    targets and also fix existing warnings. (Hong Tang via yhemanth)

    MAPREDUCE-1494. Ensure TestJobDirCleanup verifies the correct paths.
    (Amareshwari Sriramadasu via cdouglas)

    MAPREDUCE-1622. Include missing slf4j dependencies. (cdouglas)

    MAPREDUCE-1515. Accept java5.home from build.properties, not only from the
    command line when building forrest docs. (Al Thompson via cdouglas)

    MAPREDUCE-1618. Add missing javadoc to JobStatus::*JobAcls. (Amareshwari
    Sriramadasu via cdouglas)

    MAPREDUCE-1219. Remove job level metrics from jobtracker metrics to ease 
    undue load on jobtracker. (Sreekanth Ramakrishnan via sharad)

    MAPREDUCE-1604. Add Forrest documentation for Job ACLs.
    (Amareshwari Sriramadasu via yhemanth)

    MAPREDUCE-1705. Archiving and Purging of HDFS parity files should 
    handle globbed policies accurately. (Rodrigo Schmidt via dhruba)

    MAPREDUCE-1612. job conf file is not accessible from job history web page.
    (Ravi Gummadi and Sreekanth Ramakrishnan via vinodkv)

    MAPREDUCE-1397. NullPointerException observed during task failures.
    (Amareshwari Sriramadasu via vinodkv)

    MAPREDUCE-1728. Oracle timezone strings do not match Java.
    (Aaron Kimball via tomwhite)

    MAPREDUCE-1609. TaskTracker.localizeJob should not set permissions on 
    job log directory recursively. (Amareshwari Sriramadasu via vinodkv)

    MAPREDUCE-1657. After task logs directory is deleted, tasklog servlet 
    displays wrong error message about job ACLs. (Ravi Gummadi via vinodkv)

    MAPREDUCE-1727. TestJobACLs fails after HADOOP-6686. (Ravi Gummadi via vinodkv)

    MAPREDUCE-1611. Refresh nodes and refresh queues doesnt work with service
    authorization enabled. (Amar Kamat via vinodkv)

    MAPREDUCE-1276. Correct flaws in the shuffle related to connection setup
    and failure attribution. (Amareshwari Sriramadasu via cdouglas)

    MAPREDUCE-1372. ConcurrentModificationException in JobInProgress.
    (Dick King and Amareshwari Sriramadasu via tomwhite)

    MAPREDUCE-118. Fix Job.getJobID(). (Amareshwari Sriramadasu via sharad)

    MAPREDUCE-913. TaskRunner crashes with NPE resulting in held up slots,
    UNINITIALIZED tasks and hung TaskTracker. (Amareshwari Sriramadasu and
    Sreekanth Ramakrishnan via vinodkv)

    MAPREDUCE-1725.  Fix MapReduce API incompatibilities between 0.20 and 0.21.
    (tomwhite)

    MAPREDUCE-1606. TestJobACLs may timeout as there are no slots for launching
    JOB_CLEANUP task. (Ravi Gummadi via vinodkv)

    MAPREDUCE-1765. Correct streaming documentation for StreamXmlRecordReader.
    (Corinne Chandel via amareshwari)

    MAPREDUCE-1880. Fix BigDecimal.divide(..) in the pi example.  (szetszwo)

    MAPREDUCE-1885. Revert FileSystem create method that takes CreateFlags
    (MapReduce part of HADOOP-6826). (Ravi Gummadi via tomwhite)

    MAPREDUCE-1870. Harmonize MapReduce JAR library versions with Common and
    HDFS. (tomwhite)

    MAPREDUCE-1791. Remote cluster control functionality needs JavaDocs
    improvement (Konstantin Boudnik)

    MAPREDUCE-1942. 'compile-fault-inject' should never be called directly.
    (Konstantin Boudnik)

    MAPREDUCE-1876. Fixes TaskAttemptStartedEvent to correctly log event type
    for all task types. (Amar Kamat via amareshwari)

    MAPREDUCE-1926. MapReduce distribution is missing build-utils.xml.
    (tomwhite)

    MAPREDUCE-2012. Some contrib tests fail in branch 0.21 and trunk.
    (Amareshwari Sriramadasu via tomwhite)

    MAPREDUCE-1980. Fixes TaskAttemptUnsuccessfulCompletionEvent and
    TaskAttemptFinishedEvent to correctly log event type for all task types.
    (Amar Kamat via amareshwari)

    MAPREDUCE-1856. Extract a subset of tests for smoke (DOA) validation (cos)