kafka.log

Log

class Log extends Logging with KafkaMetricsGroup

An append-only log for storing messages.

The log is a sequence of LogSegments, each with a base offset denoting the first message in the segment.

New log segments are created according to a configurable policy that controls the size in bytes or time interval for a given segment.

Annotations
@threadsafe()
Linear Supertypes
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Log
  2. KafkaMetricsGroup
  3. Logging
  4. AnyRef
  5. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Log(dir: File, config: LogConfig, recoveryPoint: Long = 0L, scheduler: Scheduler, time: Time = kafka.utils.SystemTime)

    dir

    The directory in which log segments are created.

    config

    The log configuration settings

    recoveryPoint

    The offset at which to begin recovery--i.e. the first offset which has not been flushed to disk

    scheduler

    The thread pool scheduler used for background actions

    time

    The time instance used for checking the clock

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. def activeSegment: LogSegment

    The active segment that is currently taking appends

  7. def addSegment(segment: LogSegment): LogSegment

    Add the given segment to the segments in this log.

    Add the given segment to the segments in this log. If this segment replaces an existing segment, delete it.

    segment

    The segment to add

  8. def append(messages: ByteBufferMessageSet, assignOffsets: Boolean = true): LogAppendInfo

    Append this message set to the active segment of the log, rolling over to a fresh segment if necessary.

    Append this message set to the active segment of the log, rolling over to a fresh segment if necessary.

    This method will generally be responsible for assigning offsets to the messages, however if the assignOffsets=false flag is passed we will only check that the existing offsets are valid.

    messages

    The message set to append

    assignOffsets

    Should the log assign offsets to this message set or blindly apply what it is given

    returns

    Information about the appended messages including the first and last offset.

    Exceptions thrown
    KafkaStorageException

    If the append fails due to an I/O error.

  9. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  10. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  11. def close(): Unit

    Close this log

  12. var config: LogConfig

    The log configuration settings

  13. def convertToOffsetMetadata(offset: Long): LogOffsetMetadata

    Given a message offset, find its corresponding offset metadata in the log.

    Given a message offset, find its corresponding offset metadata in the log. If the message offset is out of range, return unknown offset metadata

  14. def debug(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  15. def debug(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  16. def debug(msg: ⇒ String): Unit

    Definition Classes
    Logging
  17. def deleteOldSegments(predicate: (LogSegment) ⇒ Boolean): Int

    Delete any log segments matching the given predicate function, starting with the oldest segment and moving forward until a segment doesn't match.

    Delete any log segments matching the given predicate function, starting with the oldest segment and moving forward until a segment doesn't match.

    predicate

    A function that takes in a single log segment and returns true iff it is deletable

    returns

    The number of segments deleted

  18. val dir: File

    The directory in which log segments are created.

  19. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  20. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  21. def error(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  22. def error(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  23. def error(msg: ⇒ String): Unit

    Definition Classes
    Logging
  24. def fatal(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  25. def fatal(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  26. def fatal(msg: ⇒ String): Unit

    Definition Classes
    Logging
  27. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  28. def flush(offset: Long): Unit

    Flush log segments for all offsets up to offset-1

    Flush log segments for all offsets up to offset-1

    offset

    The offset to flush up to (non-inclusive); the new recovery point

  29. def flush(): Unit

    Flush all log segments

  30. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  31. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  32. def info(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  33. def info(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  34. def info(msg: ⇒ String): Unit

    Definition Classes
    Logging
  35. def initFileSize(): Int

  36. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  37. def lastFlushTime(): Long

    The time this log is last known to have been fully flushed to disk

  38. def logEndOffset: Long

    The offset of the next message that will be appended to the log

  39. def logEndOffsetMetadata: LogOffsetMetadata

    The offset metadata of the next message that will be appended to the log

  40. var logIdent: String

    Attributes
    protected
    Definition Classes
    Logging
  41. def logSegments(from: Long, to: Long): Iterable[LogSegment]

    Get all segments beginning with the segment that includes "from" and ending with the segment that includes up to "to-1" or the end of the log (if to > logEndOffset)

  42. def logSegments: Iterable[LogSegment]

    All the log segments in this log ordered from oldest to newest

  43. def logStartOffset: Long

    The earliest message offset in the log

  44. lazy val logger: Logger

    Definition Classes
    Logging
  45. val loggerName: String

    Definition Classes
    Logging
  46. def name: String

    The name of this log

  47. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  48. def newGauge[T](name: String, metric: Gauge[T], tags: Map[String, String] = Map.empty): Gauge[T]

    Definition Classes
    KafkaMetricsGroup
  49. def newHistogram(name: String, biased: Boolean = true, tags: Map[String, String] = Map.empty): Histogram

    Definition Classes
    KafkaMetricsGroup
  50. def newMeter(name: String, eventType: String, timeUnit: TimeUnit, tags: Map[String, String] = Map.empty): Meter

    Definition Classes
    KafkaMetricsGroup
  51. def newTimer(name: String, durationUnit: TimeUnit, rateUnit: TimeUnit, tags: Map[String, String] = Map.empty): Timer

    Definition Classes
    KafkaMetricsGroup
  52. var nextOffsetMetadata: LogOffsetMetadata

  53. final def notify(): Unit

    Definition Classes
    AnyRef
  54. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  55. def numberOfSegments: Int

    The number of segments in the log.

    The number of segments in the log. Take care! this is an O(n) operation.

  56. def read(startOffset: Long, maxLength: Int, maxOffset: Option[Long] = None): FetchDataInfo

    Read messages from the log

    Read messages from the log

    startOffset

    The offset to begin reading at

    maxLength

    The maximum number of bytes to read

    maxOffset

    -The offset to read up to, exclusive. (i.e. the first offset NOT included in the resulting message set).

    returns

    The fetch data information including fetch starting offset metadata and messages read

    Exceptions thrown
    OffsetOutOfRangeException

    If startOffset is beyond the log end offset or before the base offset of the first segment.

  57. var recoveryPoint: Long

    The offset at which to begin recovery--i.

    The offset at which to begin recovery--i.e. the first offset which has not been flushed to disk

  58. def removeMetric(name: String, tags: Map[String, String] = Map.empty): Unit

    Definition Classes
    KafkaMetricsGroup
  59. def roll(): LogSegment

    Roll the log over to a new active segment starting with the current logEndOffset.

    Roll the log over to a new active segment starting with the current logEndOffset. This will trim the index to the exact size of the number of entries it currently contains.

    returns

    The newly rolled segment

  60. def size: Long

    The size of the log in bytes

  61. def swallow(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  62. def swallowDebug(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  63. def swallowError(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  64. def swallowInfo(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  65. def swallowTrace(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  66. def swallowWarn(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  67. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  68. val tags: Map[String, String]

  69. def toString(): String

    Definition Classes
    Log → AnyRef → Any
  70. val topicAndPartition: TopicAndPartition

  71. def trace(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  72. def trace(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  73. def trace(msg: ⇒ String): Unit

    Definition Classes
    Logging
  74. def unflushedMessages(): Long

    The number of messages appended to the log since the last flush

  75. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  76. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  77. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  78. def warn(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  79. def warn(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  80. def warn(msg: ⇒ String): Unit

    Definition Classes
    Logging

Inherited from KafkaMetricsGroup

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped