Type Members

1.  class KafkaContextInitializer[K, V] extends BasicContextInitializer[ConsumerRecord[K, V]]

   KafkaContextInitializer initializes Context variables with data provided in raw kafka event (see org.apache.kafka.clients.consumer.ConsumerRecord).

   KafkaContextInitializer initializes Context variables with data provided in raw kafka event (see org.apache.kafka.clients.consumer.ConsumerRecord). It is used when flink source function produces stream of ConsumerRecord (deserialized to proper key-value data types). Produces pl.touk.nussknacker.engine.api.Context with two variables: - default "input" variable which is set up with ConsumerRecord.value - metadata of kafka event, see InputMeta

   K

   - type of key of deserialized ConsumerRecord

   V

   - type of value of deserialized ConsumerRecord

2.  class KafkaSourceFactory[K, V] extends SourceFactory with SingleInputDynamicComponent[Source] with WithCachedTopicsExistenceValidator with WithExplicitTypesToExtract with UnboundedStreamComponent

   Base factory for Kafka sources with additional metadata variable.

   Base factory for Kafka sources with additional metadata variable. It is based on pl.touk.nussknacker.engine.api.context.transformation.SingleInputDynamicComponent that allows custom ValidationContext and Context transformations, which are provided by KafkaContextInitializer Can be used for single- or multi- topic sources (as csv, see topicNameSeparator and extractTopics).

   Features:

   • fetch latest N records which can be later used to test process in UI Fetching data is defined in source which may extends pl.touk.nussknacker.engine.api.process.TestDataGenerator. See pl.touk.nussknacker.engine.kafka.KafkaUtils#readLastMessages
   • reset Kafka's offset to latest value - forceLatestRead property, see pl.touk.nussknacker.engine.kafka.KafkaUtils#setOffsetToLatest

   K

   - type of key of kafka event that is generated by raw source (SourceFunction).

   V

   - type of value of kafka event that is generated by raw source (SourceFunction).