Chapter 3
第 3 章
Kafka producers are used to write messages to Kafka.
Kafka 生产者用于向 Kafka 写入消息。
ProducerConfig class helps to set configuration for the Kafka Producer.
ProducerConfig 类有助于设置 Kafka 生产者的配置。
The buffer.memory controls the total amount of memory available to the producer for buffering.
buffer.memory 控制生产者可用于缓冲的总内存量。
Topic, Producer, Consumer are the elements of Kafka.
Topic(主题)、Producer(生产者)、Consumer(消费者)是 Kafka 的要素。
The message is sent and producer waits for acknowledgement of first message to send second message. This message is sent as Synchronously.
消息发送后,生产者等待第一条消息的确认,然后再发送第二条消息。这种消息是同步发送的。
Kafka producer acks property 1 means acknowledgments the producer requires the leader to have received before considering a request complete.
Kafka 生产者的 acks 属性为 1 意味着生产者在认为请求完成之前,需要 Leader 接收到的确认。
Messages read by a Kafka consumer from a topic with multiple partitions are only ordered within single partition.
Kafka 消费者从具有多个分区的 Topic 读取的消息仅在单个分区内有序。
producer.send(record).get() is used to send a message synchronously by using a Kafka producer.
producer.send(record).get() 用于通过 Kafka 生产者同步发送消息。
The send() method is asynchronous.
send() 方法是异步的。
The acks config controls the criteria under which requests are considered complete.
acks 配置控制请求被视为完成的条件。
After creating a KafkaProducer you must close() it to avoid resource leaks.
创建 KafkaProducer 后,必须调用 close() 关闭它以避免资源泄漏。
A Kafka producer is an application that can act as a source of data in a Kafka cluster. A producer can publish messages to one or more Kafka topics.
Kafka 生产者是一个可以充当 Kafka 集群数据源的应用程序。生产者可以将消息发布到一个或多个 Kafka Topic。
Kafka producers send records to topics. The records are sometimes referred to as messages. The producer picks which partition to send a record to per topic. The producer can send records round-robin.
Kafka 生产者将记录发送到 Topic。这些记录有时被称为消息。生产者选择将记录发送到 Topic 的哪个分区。生产者可以轮询发送记录。
We create 3 partition and 3 replications of a topic. Now partitions and replications will save in different broker.
我们创建了一个 Topic 的 3 个分区和 3 个副本。现在分区和副本将保存在不同的 Broker 中。
All the partitions will divide in 3 brokers.
所有分区将分布在 3 个 Broker 中。

Explanation: - partition 1 is save in broker 1. Partition 2 is saved in broker 2. Partition 3 save in broker 3. (Round Robin Fashion).
解释:分区 1 保存在 Broker 1 中。分区 2 保存在 Broker 2 中。分区 3 保存在 Broker 3 中。(轮询方式)。
We have 2 producers here. Producer 1 produce message in partition 1 and partition 2. Producer 2 producer message in partition 3.
这里有两个生产者。生产者 1 在分区 1 和分区 2 中生成消息。生产者 2 在分区 3 中生成消息。
Overview of Kafka producer Components / Kafka 生产者组件概述
Kafka producer components are as follows:
Kafka 生产者组件如下:
Topic: A topic is a category or feed name to which records are published. Topics in Kafka are always multi-subscriber; that is, a topic can have zero, one, or many consumers that subscribe to the data written to it.
Topic(主题):Topic 是记录发布的类别或提要名称。Kafka 中的 Topic 总是多订阅者的;也就是说,一个 Topic 可以有零个、一个或多个消费者订阅写入其中的数据。
Partition: Kafka topics are divided into a number of partitions. Partitions allow you to parallelize a topic by splitting the data in a particular topic across multiple brokers. Each partition can be placed on a separate machine to allow for multiple consumers to read from a topic in parallel.
Partition(分区):Kafka Topic 被分为若干个分区。分区允许通过将特定 Topic 中的数据拆分到多个 Broker 上来并行化 Topic。每个分区可以放置在单独的机器上,以允许从多个消费者并行读取 Topic。
Serializer: the process of converting an object into a stream of bytes for the purpose of transmission is what we call Serialization. Apache Kafka offers the ability that we can easily publish as well as subscribe to streams of records. Hence, we have the flexibility to create our own custom serializer as well as reserialize which helps to transmit different data type using it.
Serializer(序列化器):将对象转换为字节流以进行传输的过程称为序列化。Apache Kafka 提供了我们可以轻松发布和订阅记录流的能力。因此,我们可以灵活地创建自己的自定义序列化器以及重新序列化,这有助于使用它传输不同的数据类型。
Broker: A broker is a stateless Kafka server. A Kafka cluster is made up of multiple Kafka Brokers. Kafka producer and consumer don’t interact directly but use the Kafka server as an agent or broker to exchange message services. Kafka cluster typically consists of multiple brokers to maintain load balance.
Broker(代理):Broker 是无状态的 Kafka 服务器。Kafka 集群由多个 Kafka Broker 组成。Kafka 生产者和消费者不直接交互,而是使用 Kafka 服务器作为代理或经纪人来交换消息服务。Kafka 集群通常由多个 Broker 组成以维持负载平衡。
Producer API / 生产者 API
We have 2 classes in kafka producer.
Kafka 生产者中有 2 个类。
Kafka Producer - to make producer
Kafka Producer - 用于创建生产者
KafkaRecord - to make the messages
KafkaRecord - 用于创建消息
Properties are an object: With the help of this object, we will set information of bootstrap server address, key serializer, value serializer.
Properties 是一个对象:借助此对象,我们将设置引导服务器地址、键序列化器、值序列化器的信息。
1 | Properties props = new Properties(); |
Bootstrap.servers: A list of host/port pairs to use for establishing the initial connection to the Kafka cluster. The client will make use of all servers irrespective of which servers are specified here for bootstrapping-this list only impacts the initial hosts used to discover the full set of servers.
Bootstrap.servers:用于建立与 Kafka 集群的初始连接的主机/端口对列表。客户端将使用所有服务器,无论此处指定了哪些服务器用于引导——此列表仅影响用于发现完整服务器集的初始主机。
Serializer: - Serialization is the process of converting an object into stream of byte.
Serializer(序列化器):- 序列化是将对象转换为字节流的过程。
key.serializer: - Serializer class for key that implements the org.apache.kafka.common.serialization.Serializer.
key.serializer:- 实现 org.apache.kafka.common.serialization.Serializer 的键(key)的序列化器类。
value.serializer: - Serializer class for value that implements the org.apache.kafka.common.serialization.Serializer.
value.serializer:- 实现 org.apache.kafka.common.serialization.Serializer 的值(value)的序列化器类。
Kafka Producer / Kafka 生产者
We have to create object for KafkaProducer class.
我们必须为 KafkaProducer 类创建对象。
1 | KafkaProducer<String, String> producer = new KafkaProducer<>(props); |
In this object, we will pass Kafka information.
在这个对象中,我们将传递 Kafka 信息。
1:- Kafka server address.
1:- Kafka 服务器地址。
2: - key serializer.
2:- 键序列化器。
3: - value serializer.
3:- 值序列化器。
KafkaRecord
1 | ProducerRecord<String, String> record = new ProducerRecord<>(topicName, key, value) |
In this object we will pass message information.
在这个对象中,我们将传递消息信息。
1: topic name.
1:Topic 名称。
2: - key
2:- 键(key)
3: - value (message).
3:- 值(消息)。
1 | producer.send(record); |