Package org.apache.flink.optimizer.dag
Class DataSourceNode
- java.lang.Object
-
- org.apache.flink.optimizer.dag.OptimizerNode
-
- org.apache.flink.optimizer.dag.DataSourceNode
-
- All Implemented Interfaces:
EstimateProvider,DumpableNode<OptimizerNode>,org.apache.flink.util.Visitable<OptimizerNode>
public class DataSourceNode extends OptimizerNode
The optimizer's internal representation of a data source.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.flink.optimizer.dag.OptimizerNode
OptimizerNode.UnclosedBranchDescriptor
-
-
Field Summary
-
Fields inherited from class org.apache.flink.optimizer.dag.OptimizerNode
cachedPlans, closedBranchingNodes, costWeight, estimatedNumRecords, estimatedOutputSize, hereJoinedBranches, id, MAX_DYNAMIC_PATH_COST_WEIGHT, onDynamicPath, openBranches, uniqueFields
-
-
Constructor Summary
Constructors Constructor Description DataSourceNode(org.apache.flink.api.common.operators.GenericDataSourceBase<?,?> pactContract)Creates a new DataSourceNode for the given contract.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidaccept(org.apache.flink.util.Visitor<OptimizerNode> visitor)This method implements the visit of a depth-first graph traversing visitor.voidcomputeInterestingPropertiesForInputs(CostEstimator estimator)Tells the node to compute the interesting properties for its inputs.protected voidcomputeOperatorSpecificDefaultEstimates(DataStatistics statistics)voidcomputeUnclosedBranchStack()This method causes the node to compute the description of open branches in its sub-plan.List<PlanNode>getAlternativePlans(CostEstimator estimator)Computes the plan alternatives for this node, an implicitly for all nodes that are children of this node.List<DagConnection>getIncomingConnections()Gets all incoming connections of this node.org.apache.flink.api.common.operators.GenericDataSourceBase<?,?>getOperator()Gets the contract object for this data source node.StringgetOperatorName()Gets the name of this node, which is the name of the function/operator, or data source / data sink.org.apache.flink.api.common.operators.SemanticPropertiesgetSemanticProperties()voidsetInput(Map<org.apache.flink.api.common.operators.Operator<?>,OptimizerNode> contractToNode, org.apache.flink.api.common.ExecutionMode defaultDataExchangeMode)This function connects the predecessors to this operator.voidsetParallelism(int parallelism)Sets the parallelism for this optimizer node.-
Methods inherited from class org.apache.flink.optimizer.dag.OptimizerNode
addBroadcastConnection, addClosedBranch, addClosedBranches, addOutgoingConnection, areBranchCompatible, clearInterestingProperties, computeOutputEstimates, computeUnclosedBranchStackForBroadcastInputs, computeUnionOfInterestingPropertiesFromSuccessors, getBranchesForParent, getBroadcastConnectionNames, getBroadcastConnections, getClosedBranchingNodes, getCostWeight, getDumpableInputs, getEstimatedAvgWidthPerOutputRecord, getEstimatedNumRecords, getEstimatedOutputSize, getId, getInterestingProperties, getMaxDepth, getMinimalMemoryAcrossAllSubTasks, getOpenBranches, getOptimizerNode, getOutgoingConnections, getParallelism, getPlanNode, getPredecessors, getUniqueFields, hasUnclosedBranches, haveAllOutputConnectionInterestingProperties, identifyDynamicPath, initId, isBranching, isOnDynamicPath, markAllOutgoingConnectionsAsPipelineBreaking, mergeLists, prunePlanAlternatives, prunePlanAlternativesWithCommonBranching, readStubAnnotations, readUniqueFieldsAnnotation, removeClosedBranches, setBroadcastInputs, setEstimatedNumRecords, setEstimatedOutputSize, toString
-
-
-
-
Method Detail
-
getOperator
public org.apache.flink.api.common.operators.GenericDataSourceBase<?,?> getOperator()
Gets the contract object for this data source node.- Overrides:
getOperatorin classOptimizerNode- Returns:
- The contract.
-
getOperatorName
public String getOperatorName()
Description copied from class:OptimizerNodeGets the name of this node, which is the name of the function/operator, or data source / data sink.- Specified by:
getOperatorNamein classOptimizerNode- Returns:
- The node name.
-
setParallelism
public void setParallelism(int parallelism)
Description copied from class:OptimizerNodeSets the parallelism for this optimizer node. The parallelism denotes how many parallel instances of the operator will be spawned during the execution.- Overrides:
setParallelismin classOptimizerNode- Parameters:
parallelism- The parallelism to set. If this value isExecutionConfig.PARALLELISM_DEFAULTthen the system will take the default number of parallel instances.
-
getIncomingConnections
public List<DagConnection> getIncomingConnections()
Description copied from class:OptimizerNodeGets all incoming connections of this node. This method needs to be overridden by subclasses to return the children.- Specified by:
getIncomingConnectionsin classOptimizerNode- Returns:
- The list of incoming connections.
-
setInput
public void setInput(Map<org.apache.flink.api.common.operators.Operator<?>,OptimizerNode> contractToNode, org.apache.flink.api.common.ExecutionMode defaultDataExchangeMode)
Description copied from class:OptimizerNodeThis function connects the predecessors to this operator.- Specified by:
setInputin classOptimizerNode- Parameters:
contractToNode- The map from program operators to optimizer nodes.defaultDataExchangeMode- The data exchange mode to use, if the operator does not specify one.
-
computeOperatorSpecificDefaultEstimates
protected void computeOperatorSpecificDefaultEstimates(DataStatistics statistics)
- Specified by:
computeOperatorSpecificDefaultEstimatesin classOptimizerNode
-
computeInterestingPropertiesForInputs
public void computeInterestingPropertiesForInputs(CostEstimator estimator)
Description copied from class:OptimizerNodeTells the node to compute the interesting properties for its inputs. The interesting properties for the node itself must have been computed before. The node must then see how many of interesting properties it preserves and add its own.- Specified by:
computeInterestingPropertiesForInputsin classOptimizerNode- Parameters:
estimator- TheCostEstimatorinstance to use for plan cost estimation.
-
computeUnclosedBranchStack
public void computeUnclosedBranchStack()
Description copied from class:OptimizerNodeThis method causes the node to compute the description of open branches in its sub-plan. An open branch describes, that a (transitive) child node had multiple outputs, which have not all been re-joined in the sub-plan. This method needs to set theopenBranchesfield to a stack of unclosed branches, the latest one top. A branch is considered closed, if some later node sees all of the branching node's outputs, no matter if there have been more branches to different paths in the meantime.- Specified by:
computeUnclosedBranchStackin classOptimizerNode
-
getAlternativePlans
public List<PlanNode> getAlternativePlans(CostEstimator estimator)
Description copied from class:OptimizerNodeComputes the plan alternatives for this node, an implicitly for all nodes that are children of this node. This method must determine for each alternative the global and local properties and the costs. This method may recursively callgetAlternatives()on its children to get their plan alternatives, and build its own alternatives on top of those.- Specified by:
getAlternativePlansin classOptimizerNode- Parameters:
estimator- The cost estimator used to estimate the costs of each plan alternative.- Returns:
- A list containing all plan alternatives.
-
getSemanticProperties
public org.apache.flink.api.common.operators.SemanticProperties getSemanticProperties()
- Specified by:
getSemanticPropertiesin classOptimizerNode
-
accept
public void accept(org.apache.flink.util.Visitor<OptimizerNode> visitor)
Description copied from class:OptimizerNodeThis method implements the visit of a depth-first graph traversing visitor. Implementers must first call thepreVisit()method, then hand the visitor to their children, and finally call thepostVisit()method.- Specified by:
acceptin interfaceorg.apache.flink.util.Visitable<OptimizerNode>- Specified by:
acceptin classOptimizerNode- Parameters:
visitor- The graph traversing visitor.- See Also:
Visitable.accept(org.apache.flink.util.Visitor)
-
-