org.apache.spark.scheduler.cluster

Class YarnSchedulerBackend

Object

org.apache.spark.scheduler.cluster.CoarseGrainedSchedulerBackend

org.apache.spark.scheduler.cluster.YarnSchedulerBackend

All Implemented Interfaces:

ExecutorAllocationClient, Logging, SchedulerBackend

public abstract class YarnSchedulerBackend
extends CoarseGrainedSchedulerBackend

Abstract Yarn scheduler backend that contains common logic between the client and cluster Yarn scheduler backends.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.spark.scheduler.cluster.CoarseGrainedSchedulerBackend

CoarseGrainedSchedulerBackend.DriverActor

Constructor Summary

Constructors

Constructor and Description
YarnSchedulerBackend(TaskSchedulerImpl scheduler, SparkContext sc)

Method Summary

Methods

Modifier and Type	Method and Description
static String	ACTOR_NAME()
boolean	doKillExecutors(scala.collection.Seq<String> executorIds) Request that the ApplicationMaster kill the specified executors.
boolean	doRequestTotalExecutors(int requestedTotal) Request executors from the ApplicationMaster by specifying the total number desired.
boolean	sufficientResourcesRegistered()

Methods inherited from class org.apache.spark.scheduler.cluster.CoarseGrainedSchedulerBackend

actorSystem, conf, createTime, defaultParallelism, driverActor, isReady, killExecutors, killTask, maxRegisteredWaitingTime, minRegisteredRatio, numExistingExecutors, removeExecutor, requestExecutors, reviveOffers, start, stop, stopExecutors, taskIdsOnSlave, totalCoreCount, totalRegisteredExecutors

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.ExecutorAllocationClient

killExecutor

Methods inherited from interface org.apache.spark.scheduler.SchedulerBackend

appId, applicationId

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

YarnSchedulerBackend

public YarnSchedulerBackend(TaskSchedulerImpl scheduler,
                    SparkContext sc)

Method Detail

ACTOR_NAME

public static String ACTOR_NAME()

doRequestTotalExecutors

public boolean doRequestTotalExecutors(int requestedTotal)

Request executors from the ApplicationMaster by specifying the total number desired. This includes executors already pending or running.

doKillExecutors

public boolean doKillExecutors(scala.collection.Seq<String> executorIds)

Request that the ApplicationMaster kill the specified executors.

sufficientResourcesRegistered

public boolean sufficientResourcesRegistered()

Overrides:

sufficientResourcesRegistered in class CoarseGrainedSchedulerBackend