The ResourceManager responds with a new, monotonically
increasing, {@link ApplicationId} which is used by the client to submit
a new application.
The ResourceManager also responds with details such
as maximum resource capabilities in the cluster as specified in
{@link GetNewApplicationResponse}.
ApplicationId
@return response containing the new ApplicationId to be used
to submit an application
@throws YarnException
@throws IOException
@see #submitApplication(SubmitApplicationRequest)]]>
ResourceManager.
The client is required to provide details such as queue,
{@link Resource} required to run the ApplicationMaster,
the equivalent of {@link ContainerLaunchContext} for launching
the ApplicationMaster etc. via the
{@link SubmitApplicationRequest}.
Currently the ResourceManager sends an immediate (empty)
{@link SubmitApplicationResponse} on accepting the submission and throws
an exception if it rejects the submission. However, this call needs to be
followed by {@link #getApplicationReport(GetApplicationReportRequest)}
to make sure that the application gets properly submitted - obtaining a
{@link SubmitApplicationResponse} from ResourceManager doesn't guarantee
that RM 'remembers' this application beyond failover or restart. If RM
failover or RM restart happens before ResourceManager saves the
application's state successfully, the subsequent
{@link #getApplicationReport(GetApplicationReportRequest)} will throw
a {@link ApplicationNotFoundException}. The Clients need to re-submit
the application with the same {@link ApplicationSubmissionContext} when
it encounters the {@link ApplicationNotFoundException} on the
{@link #getApplicationReport(GetApplicationReportRequest)} call.
During the submission process, it checks whether the application already exists. If the application exists, it will simply return SubmitApplicationResponse
In secure mode,the ResourceManager verifies access to
queues etc. before accepting the application submission.
ResourceManager to abort submitted application.
The client, via {@link KillApplicationRequest} provides the {@link ApplicationId} of the application to be aborted.
In secure mode,the ResourceManager verifies access to the
application, queue etc. before terminating the application.
Currently, the ResourceManager returns an empty response
on success and throws an exception on rejecting the request.
ResourceManager returns an empty response
on success and throws an exception on rejecting the request
@throws YarnException
@throws IOException
@see #getQueueUserAcls(GetQueueUserAclsInfoRequest)]]>
ResourceManager.
The client, via {@link GetApplicationReportRequest} provides the {@link ApplicationId} of the application.
In secure mode,the ResourceManager verifies access to the
application, queue etc. before accepting the request.
The ResourceManager responds with a
{@link GetApplicationReportResponse} which includes the
{@link ApplicationReport} for the application.
If the user does not have VIEW_APP access then the
following fields in the report will be set to stubbed values:
ResourceManager.
The ResourceManager responds with a
{@link GetClusterMetricsResponse} which includes the
{@link YarnClusterMetrics} with details such as number of current
nodes in the cluster.
ResourceManager.
The ResourceManager responds with a
{@link GetApplicationsResponse} which includes the
{@link ApplicationReport} for the applications.
If the user does not have VIEW_APP access for an
application then the corresponding report will be filtered as
described in {@link #getApplicationReport(GetApplicationReportRequest)}.
ResourceManager.
The ResourceManager responds with a
{@link GetClusterNodesResponse} which includes the
{@link NodeReport} for all the nodes in the cluster.
ResourceManager.
The client, via {@link GetQueueInfoRequest}, can ask for details such as used/total resources, child queues, running applications etc.
In secure mode,the ResourceManager verifies access before
providing the information.
ResourceManager.
The ResourceManager responds with queue acls for all
existing queues.
The ResourceManager responds with the delegation
{@link Token} that can be used by the client to speak to this
service.
@param request request to get a delegation token for the client.
@return delegation token that can be used to talk to this service
@throws YarnException
@throws IOException]]>
ResourceManager
The client, via {@link GetApplicationAttemptReportRequest} provides the {@link ApplicationAttemptId} of the application attempt.
In secure mode,the ResourceManager verifies access to
the method before accepting the request.
The ResourceManager responds with a
{@link GetApplicationAttemptReportResponse} which includes the
{@link ApplicationAttemptReport} for the application attempt.
If the user does not have VIEW_APP access then the following
fields in the report will be set to stubbed values:
ResourceManager
The ResourceManager responds with a
{@link GetApplicationAttemptsRequest} which includes the
{@link ApplicationAttemptReport} for all the applications attempts of a
specified application attempt.
If the user does not have VIEW_APP access for an application
then the corresponding report will be filtered as described in
{@link #getApplicationAttemptReport(GetApplicationAttemptReportRequest)}.
ResourceManager
The client, via {@link GetContainerReportRequest} provides the {@link ContainerId} of the container.
In secure mode,the ResourceManager verifies access to the
method before accepting the request.
The ResourceManager responds with a
{@link GetContainerReportResponse} which includes the
{@link ContainerReport} for the container.
ResourceManager
The client, via {@link GetContainersRequest} provides the {@link ApplicationAttemptId} of the application attempt.
In secure mode,the ResourceManager verifies access to the
method before accepting the request.
The ResourceManager responds with a
{@link GetContainersResponse} which includes a list of
{@link ContainerReport} for all the containers of a specific application
attempt.
The client packages all details of its request in a {@link ReservationSubmissionRequest} object. This contains information about the amount of capacity, temporal constraints, and concurrency needs. Furthermore, the reservation might be composed of multiple stages, with ordering dependencies among them.
In order to respond, a new admission control component in the {@code ResourceManager} performs an analysis of the resources that have been committed over the period of time the user is requesting, verify that the user requests can be fulfilled, and that it respect a sharing policy (e.g., {@code CapacityOverTimePolicy}). Once it has positively determined that the ReservationSubmissionRequest is satisfiable the {@code ResourceManager} answers with a {@link ReservationSubmissionResponse} that include a non-null {@link ReservationId}. Upon failure to find a valid allocation the response is an exception with the reason. On application submission the client can use this {@link ReservationId} to obtain access to the reserved resources.
The system guarantees that during the time-range specified by the user, the reservationID will be corresponding to a valid reservation. The amount of capacity dedicated to such queue can vary overtime, depending of the allocation that has been determined. But it is guaranteed to satisfy all the constraint expressed by the user in the {@link ReservationSubmissionRequest}.
@param request the request to submit a new Reservation @return response the {@link ReservationId} on accepting the submission @throws YarnException if the request is invalid or reservation cannot be created successfully @throws IOException]]>The allocation is attempted by virtually substituting all previous allocations related to this Reservation with new ones, that satisfy the new {@link ReservationUpdateRequest}. Upon success the previous allocation is substituted by the new one, and on failure (i.e., if the system cannot find a valid allocation for the updated request), the previous allocation remains valid. The {@link ReservationId} is not changed, and applications currently running within this reservation will automatically receive the resources based on the new allocation.
@param request to update an existing Reservation (the ReservationRequest should refer to an existing valid {@link ReservationId}) @return response empty on successfully updating the existing reservation @throws YarnException if the request is invalid or reservation cannot be updated successfully @throws IOException]]>ResourceManager
to submit/abort jobs and to get information on applications, cluster metrics,
nodes, queues and ACLs.]]>
ResourceManager.
The client, via {@link GetApplicationReportRequest} provides the {@link ApplicationId} of the application.
In secure mode,the ApplicationHistoryServer verifies access to
the application, queue etc. before accepting the request.
The ApplicationHistoryServer responds with a
{@link GetApplicationReportResponse} which includes the
{@link ApplicationReport} for the application.
If the user does not have VIEW_APP access then the following
fields in the report will be set to stubbed values:
ApplicationHistoryServer.
The ApplicationHistoryServer responds with a
{@link GetApplicationsResponse} which includes a list of
{@link ApplicationReport} for all the applications.
If the user does not have VIEW_APP access for an application
then the corresponding report will be filtered as described in
{@link #getApplicationReport(GetApplicationReportRequest)}.
ApplicationHistoryServer.
The client, via {@link GetApplicationAttemptReportRequest} provides the {@link ApplicationAttemptId} of the application attempt.
In secure mode,the ApplicationHistoryServer verifies access to
the method before accepting the request.
The ApplicationHistoryServer responds with a
{@link GetApplicationAttemptReportResponse} which includes the
{@link ApplicationAttemptReport} for the application attempt.
If the user does not have VIEW_APP access then the following
fields in the report will be set to stubbed values:
ApplicationHistoryServer.
The ApplicationHistoryServer responds with a
{@link GetApplicationAttemptsRequest} which includes the
{@link ApplicationAttemptReport} for all the applications attempts of a
specified application attempt.
If the user does not have VIEW_APP access for an application
then the corresponding report will be filtered as described in
{@link #getApplicationAttemptReport(GetApplicationAttemptReportRequest)}.
ApplicationHistoryServer.
The client, via {@link GetContainerReportRequest} provides the {@link ContainerId} of the container.
In secure mode,the ApplicationHistoryServer verifies access to
the method before accepting the request.
The ApplicationHistoryServer responds with a
{@link GetContainerReportResponse} which includes the
{@link ContainerReport} for the container.
ApplciationHistoryServer.
The client, via {@link GetContainersRequest} provides the {@link ApplicationAttemptId} of the application attempt.
In secure mode,the ApplicationHistoryServer verifies access to
the method before accepting the request.
The ApplicationHistoryServer responds with a
{@link GetContainersResponse} which includes a list of
{@link ContainerReport} for all the containers of a specific application
attempt.
The ApplicationHistoryServer responds with the delegation
token {@link Token} that can be used by the client to speak to this
service.
ApplicationHistoryServer to
get the information of completed applications etc.
]]>
ApplicationMaster to register with
the ResourceManager.
The ApplicationMaster needs to provide details such as RPC
Port, HTTP tracking url etc. as specified in
{@link RegisterApplicationMasterRequest}.
The ResourceManager responds with critical details such as
maximum resource capabilities in the cluster as specified in
{@link RegisterApplicationMasterResponse}.
ApplicationMaster to notify the
ResourceManager about its completion (success or failed).
The ApplicationMaster has to provide details such as
final state, diagnostics (in case of failures) etc. as specified in
{@link FinishApplicationMasterRequest}.
The ResourceManager responds with
{@link FinishApplicationMasterResponse}.
ApplicationMaster and the
ResourceManager.
The ApplicationMaster uses this interface to provide a list of
{@link ResourceRequest} and returns unused {@link Container} allocated to
it via {@link AllocateRequest}. Optionally, the
ApplicationMaster can also blacklist resources which
it doesn't want to use.
This also doubles up as a heartbeat to let the
ResourceManager know that the ApplicationMaster
is alive. Thus, applications should periodically make this call to be kept
alive. The frequency depends on
{@link YarnConfiguration#RM_AM_EXPIRY_INTERVAL_MS} which defaults to
{@link YarnConfiguration#DEFAULT_RM_AM_EXPIRY_INTERVAL_MS}.
The ResourceManager responds with list of allocated
{@link Container}, status of completed containers and headroom information
for the application.
The ApplicationMaster can use the available headroom
(resources) to decide how to utilized allocated resources and make informed
decisions about future resource requests.
ApplicationMaster
and the ResourceManager.
This is used by the ApplicationMaster to register/unregister
and to request and obtain resources in the cluster from the
ResourceManager.
ApplicationMaster provides a list of
{@link StartContainerRequest}s to a NodeManager to
start {@link Container}s allocated to it using this interface.
The ApplicationMaster has to provide details such as allocated
resource capability, security tokens (if enabled), command to be executed
to start the container, environment for the process, necessary
binaries/jar/shared-objects etc. via the {@link ContainerLaunchContext} in
the {@link StartContainerRequest}.
The NodeManager sends a response via
{@link StartContainersResponse} which includes a list of
{@link Container}s of successfully launched {@link Container}s, a
containerId-to-exception map for each failed {@link StartContainerRequest} in
which the exception indicates errors from per container and a
allServicesMetaData map between the names of auxiliary services and their
corresponding meta-data. Note: None-container-specific exceptions will
still be thrown by the API method itself.
The ApplicationMaster can use
{@link #getContainerStatuses(GetContainerStatusesRequest)} to get updated
statuses of the to-be-launched or launched containers.
ApplicationMaster requests a NodeManager to
stop a list of {@link Container}s allocated to it using this
interface.
The ApplicationMaster sends a {@link StopContainersRequest}
which includes the {@link ContainerId}s of the containers to be stopped.
The NodeManager sends a response via
{@link StopContainersResponse} which includes a list of {@link ContainerId}
s of successfully stopped containers, a containerId-to-exception map for
each failed request in which the exception indicates errors from per
container. Note: None-container-specific exceptions will still be thrown by
the API method itself. ApplicationMaster can use
{@link #getContainerStatuses(GetContainerStatusesRequest)} to get updated
statuses of the containers.
ApplicationMaster to request for current
statuses of Containers from the NodeManager.
The ApplicationMaster sends a
{@link GetContainerStatusesRequest} which includes the {@link ContainerId}s
of all containers whose statuses are needed.
The NodeManager responds with
{@link GetContainerStatusesResponse} which includes a list of
{@link ContainerStatus} of the successfully queried containers and a
containerId-to-exception map for each failed request in which the exception
indicates errors from per container. Note: None-container-specific
exceptions will still be thrown by the API method itself.
ContainerStatuses of containers with
the specified ContainerIds
@return response containing the list of ContainerStatus of the
successfully queried containers and a containerId-to-exception map
for failed requests.
@throws YarnException
@throws IOException]]>
ApplicationMaster and a
NodeManager to start/stop containers and to get status
of running containers.
If security is enabled the NodeManager verifies that the
ApplicationMaster has truly been allocated the container
by the ResourceManager and also verifies all interactions such
as stopping the container or obtaining status information for the container.
ResourceManager about the application's resource requirements.
@return the list of ResourceRequest
@see ResourceRequest]]>
ResourceManager about the application's resource requirements.
@param resourceRequests list of ResourceRequest to update the
ResourceManager about the application's
resource requirements
@see ResourceRequest]]>
ApplicationMaster.
@return list of ContainerId of containers being
released by the ApplicationMaster]]>
ApplicationMaster
@param releaseContainers list of ContainerId of
containers being released by the
ApplicationMaster]]>
ApplicationMaster.
@return the ResourceBlacklistRequest being sent by the
ApplicationMaster
@see ResourceBlacklistRequest]]>
ResourceManager about the blacklist additions and removals
per the ApplicationMaster.
@param resourceBlacklistRequest the ResourceBlacklistRequest
to inform the ResourceManager about
the blacklist additions and removals
per the ApplicationMaster
@see ResourceBlacklistRequest]]>
ApplicationMaster]]>
ResourceManager about some container's resources need to be
increased]]>
ApplicationMaster to the
ResourceManager to obtain resources in the cluster.
The request includes:
ResourceManager about the application's
resource requirements.
ApplicationMaster to take some action then it will send an
AMCommand to the ApplicationMaster. See AMCommand
for details on commands and actions for them.
@return AMCommand if the ApplicationMaster should
take action, null otherwise
@see AMCommand]]>
Container by the
ResourceManager.
@return list of newly allocated Container]]>
NodeReports. Updates could
be changes in health, availability etc of the nodes.
@return The delta of updated nodes since the last response]]>
The message is a snapshot of the resources the RM wants back from the AM.
While demand persists, the RM will repeat its request; applications should
not interpret each message as a request for additional
resources on top of previous messages. Resources requested consistently
over some duration may be forcibly killed by the RM.
@return A specification of the resources to reclaim from this AM.]]>
1) AM is receiving first container on underlying NodeManager.
OR
2) NMToken master key rolled over in ResourceManager and AM is getting new
container on the same underlying NodeManager.
AM will receive one NMToken per NM irrespective of the number of containers issued on same NM. AM is expected to store these tokens until issued a new token for the same NM.
]]>
ResourceManager the
ApplicationMaster during resource negotiation.
The response, includes:
ApplicationMaster
take some actions (resync, shutdown etc.).
ApplicationMaster.
@return final state of the ApplicationMaster]]>
ApplicationMaster
@param finalState final state of the ApplicationMaster]]>
ApplicationMaster.
This url if contains scheme then that will be used by resource manager
web application proxy otherwise it will default to http.
@return tracking URLfor the ApplicationMaster]]>
ApplicationMaster.
This is the web-URL to which ResourceManager or web-application proxy will
redirect client/users once the application is finished and the
ApplicationMaster is gone.
If the passed url has a scheme then that will be used by the ResourceManager and web-application proxy, otherwise the scheme will default to http.
Empty, null, "N/A" strings are all valid besides a real URL. In case an url isn't explicitly passed, it defaults to "N/A" on the ResourceManager.
@param url
tracking URLfor the ApplicationMaster]]>
ApplicationMaster to
inform the ResourceManager about its completion.
The final request includes details such:
ApplicationMasterApplicationMaster
ResourceManager to a
ApplicationMaster on it's completion.
The response, includes:
ApplicationAttemptId of an application attempt]]>
ApplicationAttemptId of an application attempt]]>
ResourceManager to get an
{@link ApplicationAttemptReport} for an application attempt.
The request should include the {@link ApplicationAttemptId} of the application attempt.
@see ApplicationAttemptReport @see ApplicationHistoryProtocol#getApplicationAttemptReport(GetApplicationAttemptReportRequest)]]>ApplicationAttemptReport for the application attempt]]>
ApplicationAttemptReport for the application attempt]]>
ResourceManager to a client requesting
an application attempt report.
The response includes an {@link ApplicationAttemptReport} which has the details about the particular application attempt
@see ApplicationAttemptReport @see ApplicationHistoryProtocol#getApplicationAttemptReport(GetApplicationAttemptReportRequest)]]>ApplicationId of an application]]>
ApplicationId of an application]]>
ResourceManager.
@see ApplicationHistoryProtocol#getApplicationAttempts(GetApplicationAttemptsRequest)]]>
ApplicationReport of an application]]>
ApplicationReport of an application]]>
ResourceManager to a client requesting
a list of {@link ApplicationAttemptReport} for application attempts.
The ApplicationAttemptReport for each application includes the
details of an application attempt.
ApplicationId of the application]]>
ApplicationId of the application]]>
ResourceManager to
get an {@link ApplicationReport} for an application.
The request should include the {@link ApplicationId} of the application.
@see ApplicationClientProtocol#getApplicationReport(GetApplicationReportRequest) @see ApplicationReport]]>ApplicationReport for the application]]>
ResourceManager to a client
requesting an application report.
The response includes an {@link ApplicationReport} which has details such
as user, queue, name, host on which the ApplicationMaster is
running, RPC port, tracking URL, diagnostics, start time etc.
ResourceManager.
@see ApplicationClientProtocol#getApplications(GetApplicationsRequest)
Setting any of the parameters to null, would just disable that filter
@param scope {@link ApplicationsRequestScope} to filter by @param users list of users to filter by @param queues list of scheduler queues to filter by @param applicationTypes types of applications @param applicationTags application tags to filter by @param applicationStates application states to filter by @param startRange range of application start times to filter by @param finishRange range of application finish times to filter by @param limit number of applications to limit to @return {@link GetApplicationsRequest} to be used with {@link ApplicationClientProtocol#getApplications(GetApplicationsRequest)}]]>ResourceManager.
@param scope {@link ApplicationsRequestScope} to filter by
@see ApplicationClientProtocol#getApplications(GetApplicationsRequest)]]>
ResourceManager.
@see ApplicationClientProtocol#getApplications(GetApplicationsRequest)]]>
ResourceManager.
@see ApplicationClientProtocol#getApplications(GetApplicationsRequest)]]>
ResourceManager.
@see ApplicationClientProtocol#getApplications(GetApplicationsRequest)]]>
ResourceManager.
@see ApplicationClientProtocol#getApplications(GetApplicationsRequest)]]>
ApplicationReport for applications]]>
ResourceManager to a client
requesting an {@link ApplicationReport} for applications.
The ApplicationReport for each application includes details
such as user, queue, name, host on which the ApplicationMaster
is running, RPC port, tracking URL, diagnostics, start time etc.
ResourceManager.
Currently, this is empty.
@see ApplicationClientProtocol#getClusterMetrics(GetClusterMetricsRequest)]]>YarnClusterMetrics for the cluster]]>
ResourceManager to a client
requesting cluster metrics.@see YarnClusterMetrics @see ApplicationClientProtocol#getClusterMetrics(GetClusterMetricsRequest)]]>
ResourceManager.
The request will ask for all nodes in the given {@link NodeState}s.
@see ApplicationClientProtocol#getClusterNodes(GetClusterNodesRequest)]]>
NodeReport for all nodes in the cluster]]>
ResourceManager to a client
requesting a {@link NodeReport} for all nodes.
The NodeReport contains per-node information such as
available resources, number of containers, tracking url, rack name, health
status etc.
@see NodeReport
@see ApplicationClientProtocol#getClusterNodes(GetClusterNodesRequest)]]>
ContainerId of the Container]]>
ContainerId of the container]]>
ResourceManager to get an
{@link ContainerReport} for a container.
]]>
ContainerReport for the container]]>
ResourceManager to a client requesting
a container report.
The response includes a {@link ContainerReport} which has details of a container.
]]>ApplicationAttemptId of an application attempt]]>
ApplicationAttemptId of an application attempt]]>
ResourceManager.
@see ApplicationHistoryProtocol#getContainers(GetContainersRequest)]]>
ContainerReport for all the containers of an
application attempt]]>
ContainerReport for all the containers of
an application attempt]]>
ResourceManager to a client requesting
a list of {@link ContainerReport} for containers.
The ContainerReport for each container includes the container
details.
ContainerStatus.
@return the list of ContainerIds of containers for which to
obtain the ContainerStatus.]]>
ContainerStatus
@param containerIds
a list of ContainerIds of containers for which to
obtain the ContainerStatus]]>
ApplicationMaster to the
NodeManager to get {@link ContainerStatus} of requested
containers.
@see ContainerManagementProtocol#getContainerStatuses(GetContainerStatusesRequest)]]>
ContainerStatuses of the requested containers.]]>
NodeManager to the
ApplicationMaster when asked to obtain the
ContainerStatus of requested containers.
@see ContainerManagementProtocol#getContainerStatuses(GetContainerStatusesRequest)]]>
Currently, this is empty.
@see ApplicationClientProtocol#getNewApplication(GetNewApplicationRequest)]]>ApplicationId allocated by the
ResourceManager.
@return new ApplicationId allocated by the
ResourceManager]]>
ResourceManager to the client for
a request to get a new {@link ApplicationId} for submitting applications.
Clients can submit an application with the returned {@link ApplicationId}.
@see ApplicationClientProtocol#getNewApplication(GetNewApplicationRequest)]]>true if applications' information is to be included,
else false]]>
true if information about child queues is required,
else false]]>
true if information about entire hierarchy is
required, false otherwise]]>
ResourceManager.
@see ApplicationClientProtocol#getQueueInfo(GetQueueInfoRequest)]]>
QueueInfo for the specified queue]]>
ResourceManager to a client
requesting information about queues in the system.
The response includes a {@link QueueInfo} which has details such as queue name, used/total capacities, running applications, child queues etc .
@see QueueInfo @see ApplicationClientProtocol#getQueueInfo(GetQueueInfoRequest)]]>ResourceManager to
get queue acls for the current user.
Currently, this is empty.
@see ApplicationClientProtocol#getQueueUserAcls(GetQueueUserAclsInfoRequest)]]>QueueUserACLInfo per queue for the user]]>
ResourceManager to clients
seeking queue acls for the user.
The response contains a list of {@link QueueUserACLInfo} which provides information about {@link QueueACL} per queue.
@see QueueACL @see QueueUserACLInfo @see ApplicationClientProtocol#getQueueUserAcls(GetQueueUserAclsInfoRequest)]]>ApplicationId of the application to be aborted]]>
ResourceManager
to abort a submitted application.
The request includes the {@link ApplicationId} of the application to be aborted.
@see ApplicationClientProtocol#forceKillApplication(KillApplicationRequest)]]>ResourceManager to the client aborting
a submitted application.
The response, includes:
ResourceManager crashes before the process of killing the
application is completed, the ResourceManager may retry this
application on recovery.
@see ApplicationClientProtocol#forceKillApplication(KillApplicationRequest)]]>
ApplicationId of the application to be moved]]>
ApplicationId of the application to be moved]]>
ResourceManager
to move a submitted application to a different queue.
The request includes the {@link ApplicationId} of the application to be moved and the queue to place it in.
@see ApplicationClientProtocol#moveApplicationAcrossQueues(MoveApplicationAcrossQueuesRequest)]]>ResourceManager to the client moving
a submitted application to a different queue.
A response without exception means that the move has completed successfully.
@see ApplicationClientProtocol#moveApplicationAcrossQueues(MoveApplicationAcrossQueuesRequest)]]>RegisterApplicationMasterRequest]]>
ApplicationMaster is
running.
@return host on which the ApplicationMaster is running]]>
ApplicationMaster is
running.
@param host host on which the ApplicationMaster
is running]]>
ApplicationMaster
is responding.
@return the RPC port on which the ApplicationMaster is
responding]]>
ApplicationMaster is
responding.
@param port RPC port on which the ApplicationMaster is
responding]]>
ApplicationMaster.
This url if contains scheme then that will be used by resource manager
web application proxy otherwise it will default to http.
@return tracking URL for the ApplicationMaster]]>
ApplicationMaster while
it is running. This is the web-URL to which ResourceManager or
web-application proxy will redirect client/users while the application and
the ApplicationMaster are still running.
If the passed url has a scheme then that will be used by the ResourceManager and web-application proxy, otherwise the scheme will default to http.
Empty, null, "N/A" strings are all valid besides a real URL. In case an url isn't explicitly passed, it defaults to "N/A" on the ResourceManager.
@param trackingUrl
tracking URLfor the ApplicationMaster]]>
ApplicationMaster to
ResourceManager on registration.
The registration includes details such as:
ApplicationACLs]]>
The ClientToAMToken master key is sent to ApplicationMaster
by ResourceManager via {@link RegisterApplicationMasterResponse}
, used to verify corresponding ClientToAMToken.
]]>
]]>
ResourceManager from previous application attempts.
@return the list of running containers as viewed by
ResourceManager from previous application attempts
@see RegisterApplicationMasterResponse#getNMTokensFromPreviousAttempts()]]>
ResourceManager to a new
ApplicationMaster on registration.
The response contains critical details such as:
ApplicationACLs for the application.NodeManager.
@return ContainerLaunchContext for the container to be started
by the NodeManager]]>
NodeManager
@param context ContainerLaunchContext for the container to be
started by the NodeManager]]>
Note: {@link NMToken} will be used for authenticating communication with NodeManager.
@return the container token to be used for authorization during starting container. @see NMToken @see ContainerManagementProtocol#startContainers(StartContainersRequest)]]>ApplicationMaster to the
NodeManager to start a container.
The ApplicationMaster has to provide details such as
allocated resource capability, security tokens (if enabled), command
to be executed to start the container, environment for the process,
necessary binaries/jar/shared-objects etc. via the
{@link ContainerLaunchContext}.
ApplicationMaster to the NodeManager to
start containers.
In each {@link StartContainerRequest}, the ApplicationMaster has
to provide details such as allocated resource capability, security tokens (if
enabled), command to be executed to start the container, environment for the
process, necessary binaries/jar/shared-objects etc. via the
{@link ContainerLaunchContext}.
ContainerId s of the containers that are
started successfully.
@see ContainerManagementProtocol#startContainers(StartContainersRequest)]]>
NodeManager.
The meta-data is returned as a Map between the auxiliary service names and
their corresponding per service meta-data as an opaque blob
ByteBuffer
To be able to interpret the per-service meta-data, you should consult the documentation for the Auxiliary-service configured on the NodeManager
@return a Map between the names of auxiliary services and their corresponding meta-data]]>NodeManager to the
ApplicationMaster when asked to start an allocated
container.
@see ContainerManagementProtocol#startContainers(StartContainersRequest)]]>
ContainerIds of containers to be stopped]]>
ContainerIds of the containers to be stopped]]>
ApplicationMaster to the
NodeManager to stop containers.
@see ContainerManagementProtocol#stopContainers(StopContainersRequest)]]>
NodeManager to the
ApplicationMaster when asked to stop allocated
containers.
@see ContainerManagementProtocol#stopContainers(StopContainersRequest)]]>
ApplicationSubmissionContext for the application]]>
ApplicationSubmissionContext for the
application]]>
ResourceManager.
The request, via {@link ApplicationSubmissionContext}, contains
details such as queue, {@link Resource} required to run the
ApplicationMaster, the equivalent of
{@link ContainerLaunchContext} for launching the
ApplicationMaster etc.
@see ApplicationClientProtocol#submitApplication(SubmitApplicationRequest)]]>
ResourceManager to a client on
application submission.
Currently, this is empty.
@see ApplicationClientProtocol#submitApplication(SubmitApplicationRequest)]]>ApplicationAttempId.
@return ApplicationId of the ApplicationAttempId]]>
Application.
@return attempt id of the Application]]>
ApplicationAttemptId denotes the particular attempt
of an ApplicationMaster for a given {@link ApplicationId}.
Multiple attempts might be needed to run an application to completion due
to temporal failures of the ApplicationMaster such as hardware
failures, connectivity issues etc. on the node on which it was scheduled.
ApplicationMaster.
@return RPC port of this attempt ApplicationMaster]]>
ApplicationMaster is running.
@return host on which this attempt of
ApplicationMaster is running]]>
ApplicationAttemptId of the attempt]]>
ContainerId of the attempt]]>
ApplicationAttemptReport is a report of an application attempt.
It includes details such as:
ApplicationMaster of this attempt is
running.ApplicationMaster of this attempt.ResourceManager.
@return short integer identifier of the ApplicationId]]>
ResourceManager which is
used to generate globally unique ApplicationId.
@return start time of the ResourceManager]]>
ApplicationId represents the globally unique
identifier for an application.
The globally unique nature of the identifier is achieved by using the
cluster timestamp i.e. start-time of the
ResourceManager along with a monotonically increasing counter
for the application.
ApplicationId of the application]]>
ApplicationAttemptId of the attempt]]>
ApplicationMaster
is running.
@return host on which the ApplicationMaster
is running]]>
ApplicationMaster.
@return RPC port of the ApplicationMaster]]>
ApplicationMaster.
ClientToAMToken is the security token used by the AMs to verify
authenticity of any client.
The ResourceManager, provides a secure token (via
{@link ApplicationReport#getClientToAMToken()}) which is verified by the
ApplicationMaster when the client directly talks to an AM.
ApplicationMaster]]>
YarnApplicationState of the application]]>
ApplicationReport is a report of an application.
It includes details such as:
ApplicationMaster is running.ApplicationMaster.Resource]]>
Resource]]>
Resource]]>
ApplicationId of the submitted application]]>
ApplicationId of the submitted
application]]>
Priority of the application]]>
Container with which the ApplicationMaster is
launched.
@return ContainerLaunchContext for the
ApplicationMaster container]]>
Container with which the ApplicationMaster is
launched.
@param amContainer ContainerLaunchContext for the
ApplicationMaster container]]>
ApplicationMaster for
this application.]]>
ApplicationMaster
for this application.]]>
LogAggregationContext of the application]]>
ApplicationSubmissionContext represents all of the
information needed by the ResourceManager to launch
the ApplicationMaster for an application.
It includes details such as:
ApplicationMaster is executed.
Resource allocated to the container]]>
Container was
allocated.
@return Priority at which the Container was
allocated]]>
ContainerToken is the security token used by the framework
to verify authenticity of any Container.
The ResourceManager, on container allocation provides a
secure token which is verified by the NodeManager on
container launch.
Applications do not need to care about ContainerToken, they
are transparently handled by the framework - the allocated
Container includes the ContainerToken.
ContainerToken for the container]]>
Container represents an allocated resource in the cluster.
The ResourceManager is the sole authority to allocate any
Container to applications. The allocated Container
is always on a single node and has a unique {@link ContainerId}. It has
a specific amount of {@link Resource} allocated.
It includes details such as:
Typically, an ApplicationMaster receives the
Container from the ResourceManager during
resource-negotiation and then talks to the NodeManager to
start/stop containers.
Container was assigned.
Note: If containers are kept alive across application attempts via
{@link ApplicationSubmissionContext#setKeepContainersAcrossApplicationAttempts(boolean)}
the ContainerId does not necessarily contain the current
running application attempt's ApplicationAttemptId This
container can be allocated by previously exited application attempt and
managed by the current running attempt thus have the previous application
attempt's ApplicationAttemptId.
ApplicationAttemptId of the application to which the
Container was assigned]]>
getContainerId instead.
@return lower 32 bits of identifier of the ContainerId]]>
ContainerId]]>
ContainerId represents a globally unique identifier
for a {@link Container} in the cluster.]]>
LocalResource required by the container]]>
LocalResource required by the container]]>
This will be used to initialize this application on the specific {@link AuxiliaryService} running on the NodeManager by calling {@link AuxiliaryService#initializeApplication(ApplicationInitializationContext)}
@return application-specific binary service data]]>ApplicationACLs]]>
ApplicationACLs for the application]]>
ContainerLaunchContext represents all of the information
needed by the NodeManager to launch a container.
It includes details such as:
ContainerId of the container.]]>
Resource of the container.]]>
NodeId where container is running.]]>
Priority of the container.]]>
ContainerState of the container.]]>
exit status of the container.]]>
ContainerReport is a report of an container.
It includes details such as:
Container.]]>
ContainerId of the container]]>
ContainerState of the container]]>
Note: This is valid only for completed containers i.e. containers with state {@link ContainerState#COMPLETE}. Otherwise, it returns an ContainerExitStatus.INVALID.
Containers killed by the framework, either due to being released by the application or being 'lost' due to node failures etc. have a special exit code of ContainerExitStatus.ABORTED.
When threshold number of the nodemanager-local-directories or threshold number of the nodemanager-log-directories become bad, then container is not launched and is exited with ContainersExitStatus.DISKS_FAILED.
@return exit status for the container]]>ContainerStatus represents the current status of a
Container.
It provides details such as:
ContainerId of the container.ContainerState of the container.LocalResourceType of the resource to be localized]]>
LocalResourceType of the resource to be localized]]>
LocalResourceVisibility of the resource to be
localized]]>
LocalResourceVisibility of the resource to be
localized]]>
PATTERN).
@return pattern that should be used to extract entries from the
archive.]]>
PATTERN).
@param pattern pattern that should be used to extract entries
from the archive.]]>
LocalResource represents a local resource required to
run a container.
The NodeManager is responsible for localizing the resource
prior to launching the container.
Applications can specify {@link LocalResourceType} and {@link LocalResourceVisibility}.
@see LocalResourceType @see LocalResourceVisibility @see ContainerLaunchContext @see ApplicationSubmissionContext @see ContainerManagementProtocol#startContainers(org.apache.hadoop.yarn.api.protocolrecords.StartContainersRequest)]]>LocalResourceType specifies the type
of a resource localized by the NodeManager.
The type can be one of:
NodeManager.
LocalResourceVisibility specifies the visibility
of a resource localized by the NodeManager.
The visibility can be one of:
LogAggregationContext represents all of the
information needed by the NodeManager to handle
the logs for an application.
It includes details such as:
NodeManager for which the
NMToken is used to authenticate.]]>
NodeManager]]>
NodeManager
It is issued by ResourceMananger when ApplicationMaster
negotiates resource with ResourceManager and
validated on NodeManager side.
NodeId is the unique identifier for a node.
It includes the hostname and port to uniquely
identify the node. Thus, it is unique across restarts of any
NodeManager.
NodeId of the node]]>
NodeState of the node]]>
Resource on the node.
@return used Resource on the node]]>
Resource on the node.
@return total Resource on the node]]>
NodeReport is a summary of runtime information of a
node in the cluster.
It includes details such as:
Node.]]>
ResourceManager.
@see AllocateRequest#setAskList(List)]]>
ResourceManager. If the AM prefers a different set of
containers, then it may checkpoint or kill containers matching the
description in {@link #getResourceRequest}.
@return Set of containers at risk if the contract is not met.]]>
ApplicationMaster (AM) can satisfy this request according
to its own priorities to prevent containers from being forcibly killed by
the platform.
@see PreemptionMessage]]>
ApplicationMaster (AM). The AM receives a {@link
StrictPreemptionContract} message encoding which containers the platform may
forcibly kill, granting it an opportunity to checkpoint state or adjust its
execution plan. The message may also include a {@link PreemptionContract}
granting the AM more latitude in selecting which resources to return to the
cluster.
The AM should decode both parts of the message. The {@link StrictPreemptionContract} specifies particular allocations that the RM requires back. The AM can checkpoint containers' state, adjust its execution plan to move the computation, or take no action and hope that conditions that caused the RM to ask for the container will change.
In contrast, the {@link PreemptionContract} also includes a description of resources with a set of containers. If the AM releases containers matching that profile, then the containers enumerated in {@link PreemptionContract#getContainers()} may not be killed.
Each preemption message reflects the RM's current understanding of the
cluster state, so a request to return
The policy enforced by the RM is part of the scheduler. Generally, only containers that have been requested consistently should be killed, but the details are not specified.
]]>
QueueACL enumerates the various ACLs for queues.
The ACL is one of:
QueueState of the queue]]>
accessible node labels of the queue]]>
ApplicationSubmissionContext and
ResourceRequest don't specify their
NodeLabelExpression.
@return default node label expression of the queue]]>
It includes information such as:
A queue is in one of:
QueueACL for the given user]]>
QueueUserACLInfo provides information {@link QueueACL} for
the given user.
@see QueueACL
@see ApplicationClientProtocol#getQueueUserAcls(org.apache.hadoop.yarn.api.protocolrecords.GetQueueUserAclsInfoRequest)]]>
The globally unique nature of the identifier is achieved by using the cluster timestamp i.e. start-time of the {@code ResourceManager} along with a monotonically increasing counter for the reservation.
]]>It includes:
Resource models a set of computer resources in the
cluster.
Currently it models both memory and CPU.
The unit for memory is megabytes. CPU is modeled with virtual cores (vcores), a unit for expressing parallelism. A node's capacity should be configured with virtual cores equal to its number of physical cores. A container should be requested with the number of cores it can saturate, i.e. the average number of threads it expects to have runnable at a time.
Virtual cores take integer values and thus currently CPU-scheduling is very coarse. A complementary axis for CPU requests that represents processing power will likely be added in the future to enable finer-grained resource configuration.
Typically, applications request Resource of suitable
capability to run their component tasks.
Priority of the request]]>
Priority of the request]]>
Resource capability of the request]]>
Resource capability of the request]]>
ResourceRequest.]]>
If the flag is off on a rack-level ResourceRequest,
containers at that request's priority will not be assigned to nodes on that
request's rack unless requests specifically for those nodes have also been
submitted.
If the flag is off on an {@link ResourceRequest#ANY}-level
ResourceRequest, containers at that request's priority will
only be assigned on racks for which specific requests have also been
submitted.
For example, to request a container strictly on a specific node, the corresponding rack-level and any-level requests should have locality relaxation set to false. Similarly, to request a container strictly on a specific rack, the corresponding any-level request should have locality relaxation set to false.
@param relaxLocality whether locality relaxation is enabled with this
ResourceRequest.]]>
ResourceRequest represents the request made by an
application to the ResourceManager to obtain various
Container allocations.
It includes:
true,
which tells the ResourceManager if the application wants
locality to be loose (i.e. allows fall-through to rack or any)
or strict (i.e. specify hard constraint on resource allocation).
ResourceManager.
@return the set of {@link ContainerId} to be preempted.]]>
Token is the security entity used by the framework
to verify authenticity of any resource.]]>
URL represents a serializable {@link java.net.URL}.]]>
NodeManagers in the cluster]]>
YarnClusterMetrics represents cluster metrics.
Currently only number of NodeManagers is provided.
The reader and writer users/groups pattern that the user can supply is the
same as what AccessControlList takes.
Primary filters will be used to index the entities in
TimelineStore, such that users should carefully choose the
information they want to store as the primary filters. The remaining can be
stored as other information.
InetSocketAddress. On an HA cluster,
this fetches the address corresponding to the RM identified by
{@link #RM_HA_ID}.
@param name property name.
@param defaultAddress the default value
@param defaultPort the default port
@return InetSocketAddress]]>
Note: Use {@link DEFAULT_YARN_CROSS_PLATFORM_APPLICATION_CLASSPATH} for cross-platform practice i.e. submit an application from a Windows client to a Linux/Unix server or vice versa.
]]>