mapred.capacity-scheduler.maximum-system-jobs 3000 Maximum number of jobs in the system which can be initialized, concurrently, by the CapacityScheduler. mapred.capacity-scheduler.queue.default.capacity 100 Percentage of the number of slots in the cluster that are to be available for jobs in this queue. mapred.capacity-scheduler.queue.default.maximum-capacity -1 maximum-capacity defines a limit beyond which a queue cannot use the capacity of the cluster. This provides a means to limit how much excess capacity a queue can use. By default, there is no limit. The maximum-capacity of a queue can only be greater than or equal to its minimum capacity. Default value of -1 implies a queue can use complete capacity of the cluster. This property could be to curtail certain jobs which are long running in nature from occupying more than a certain percentage of the cluster, which in the absence of pre-emption, could lead to capacity guarantees of other queues being affected. One important thing to note is that maximum-capacity is a percentage , so based on the cluster's capacity the max capacity would change. So if large no of nodes or racks get added to the cluster , max Capacity in absolute terms would increase accordingly. mapred.capacity-scheduler.queue.default.supports-priority false If true, priorities of jobs will be taken into account in scheduling decisions. mapred.capacity-scheduler.queue.default.minimum-user-limit-percent 100 Each queue enforces a limit on the percentage of resources allocated to a user at any given time, if there is competition for them. This user limit can vary between a minimum and maximum value. The former depends on the number of users who have submitted jobs, and the latter is set to this property value. For example, suppose the value of this property is 25. If two users have submitted jobs to a queue, no single user can use more than 50% of the queue resources. If a third user submits a job, no single user can use more than 33% of the queue resources. With 4 or more users, no user can use more than 25% of the queue's resources. A value of 100 implies no user limits are imposed. mapred.capacity-scheduler.queue.default.user-limit-factor 1 The multiple of the queue capacity which can be configured to allow a single user to acquire more slots. mapred.capacity-scheduler.queue.default.maximum-initialized-active-tasks 200000 The maximum number of tasks, across all jobs in the queue, which can be initialized concurrently. Once the queue's jobs exceed this limit they will be queued on disk. mapred.capacity-scheduler.queue.default.maximum-initialized-active-tasks-per-user 100000 The maximum number of tasks per-user, across all the of the user's jobs in the queue, which can be initialized concurrently. Once the user's jobs exceed this limit they will be queued on disk. mapred.capacity-scheduler.queue.default.init-accept-jobs-factor 10 The multipe of (maximum-system-jobs * queue-capacity) used to determine the number of jobs which are accepted by the scheduler. mapred.capacity-scheduler.default-supports-priority false If true, priorities of jobs will be taken into account in scheduling decisions by default in a job queue. mapred.capacity-scheduler.default-minimum-user-limit-percent 100 The percentage of the resources limited to a particular user for the job queue at any given point of time by default. mapred.capacity-scheduler.default-user-limit-factor 1 The default multiple of queue-capacity which is used to determine the amount of slots a single user can consume concurrently. mapred.capacity-scheduler.default-maximum-active-tasks-per-queue 200000 The default maximum number of tasks, across all jobs in the queue, which can be initialized concurrently. Once the queue's jobs exceed this limit they will be queued on disk. mapred.capacity-scheduler.default-maximum-active-tasks-per-user 100000 The default maximum number of tasks per-user, across all the of the user's jobs in the queue, which can be initialized concurrently. Once the user's jobs exceed this limit they will be queued on disk. mapred.capacity-scheduler.default-init-accept-jobs-factor 10 The default multipe of (maximum-system-jobs * queue-capacity) used to determine the number of jobs which are accepted by the scheduler. mapred.capacity-scheduler.init-poll-interval 5000 The amount of time in miliseconds which is used to poll the job queues for jobs to initialize. mapred.capacity-scheduler.init-worker-threads 5 Number of worker threads which would be used by Initialization poller to initialize jobs in a set of queue. If number mentioned in property is equal to number of job queues then a single thread would initialize jobs in a queue. If lesser then a thread would get a set of queues assigned. If the number is greater then number of threads would be equal to number of job queues.