Package org.apache.flink.connector.base.source.reader.fetcher

Class SplitFetcher<E,SplitT extends org.apache.flink.api.connector.source.SourceSplit>

java.lang.Object

org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher<E,SplitT>

All Implemented Interfaces:

Runnable

@PublicEvolving
public class SplitFetcher<E,SplitT extends org.apache.flink.api.connector.source.SourceSplit>
extends Object
implements Runnable

The internal fetcher runnable responsible for polling message from the external system.

Method Summary

Modifier and Type	Method	Description
void	addSplits(List<SplitT> splitsToAdd)	Add splits to the split fetcher.
void	enqueueTask(SplitFetcherTask task)
int	fetcherId()
SplitReader<E,SplitT>	getSplitReader()
void	pause()
void	pauseOrResumeSplits(Collection<SplitT> splitsToPause, Collection<SplitT> splitsToResume)	Called when some splits of this source instance progressed too much beyond the global watermark of all subtasks.
void	removeSplits(List<SplitT> splitsToRemove)	Notice the split fetcher that some splits finished.
void	resume()
void	run()
void	shutdown()	Shutdown the split fetcher.
void	shutdown(boolean waitingForRecordsProcessed)	Shutdown the split fetcher.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

run

public void run()

Specified by:

run in interface Runnable

addSplits

public void addSplits(List<SplitT> splitsToAdd)

Add splits to the split fetcher. This operation is asynchronous.

Parameters:

splitsToAdd - the splits to add.

removeSplits

public void removeSplits(List<SplitT> splitsToRemove)

Notice the split fetcher that some splits finished. This operation is asynchronous.

Parameters:

splitsToRemove - the splits need to be removed.

pauseOrResumeSplits

public void pauseOrResumeSplits(Collection<SplitT> splitsToPause,
                                Collection<SplitT> splitsToResume)

Called when some splits of this source instance progressed too much beyond the global watermark of all subtasks. If the split reader implements SplitReader, it will relay the information asynchronously through the split fetcher thread.

Parameters:

splitsToPause - the splits to pause

splitsToResume - the splits to resume

enqueueTask

public void enqueueTask(SplitFetcherTask task)

getSplitReader

public SplitReader<E,SplitT> getSplitReader()

fetcherId

public int fetcherId()

shutdown

public void shutdown()

Shutdown the split fetcher.

shutdown

public void shutdown(boolean waitingForRecordsProcessed)

Shutdown the split fetcher. When waitingForRecordsProcessed is set to true, the split fetcher will block waiting for the previously emitted records to be processed before it closes the encapsulated SplitReader. Otherwise, it will just close the SplitReader.

This method is package private because it should only be used by the SplitFetcherManager when closing the idle fetchers.

Parameters:

waitingForRecordsProcessed - whether wait for the previously emitted records to be processed.

pause

public void pause()

resume

public void resume()