3节点kafka关闭leader后消费不正常，丢失接收不了消息？

关键配置： "topic": "__consumer_offsets"的副本数

/opt/kafka/kafka_2.13-3.8.0/config

在初始化搭建的时候，注意修改红色的部分内容：（3个节点）

[root@k1 config]# cat server.properties |grep -v "^$"|grep -v "#"

offsets.topic.replication.factor=3 ##默认是1，改成3

transaction.state.log.replication.factor=3 ##默认是1，改成3

如果上面的参数没改之前就搭建完成了，发现测试不正常，那么可按如下方法修改：

[root@k3 ~]# /opt/kafka/kafka_2.13-3.8.0/bin/kafka-topics.sh --create --bootstrap-server 192.168.207.131:9092,192.168.207.132:9092,192.168.207.133:9092 --replication-factor 3 --partitions 3 --topic iom-int-topic

Created topic iom-int-topic.

[root@k3 ~]# /opt/kafka/kafka_2.13-3.8.0/bin/kafka-topics.sh --list --bootstrap-server 192.168.207.131:9092,192.168.207.132:9092,192.168.207.133:9092

__consumer_offsets

iom-int-topic

但发现__consumer_offsets的ReplicationFactor还是1

的

[root@k3 ~]# /opt/kafka/kafka_2.13-3.8.0/bin/kafka-topics.sh --describe --bootstrap-server 192.168.207.131:9092,192.168.207.132:9092,192.168.207.133:9092 --topic __consumer_offsets

[2024-10-11 11:17:05,275] WARN [AdminClient clientId=adminclient-1] The DescribeTopicPartitions API is not supported, using Metadata API to describe topics. (org.apache.kafka.clients.admin.KafkaAdminClient)

Topic: __consumer_offsets TopicId: Jlmj5uIBQ0GtlG7K587Ruw PartitionCount: 50 ReplicationFactor: 1 Configs: compression.type=producer,cleanup.policy=compact,segment.bytes=104857600

Topic: __consumer_offsets Partition: 0 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 1 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 2 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 3 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 4 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 5 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 6 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 7 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 8 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 9 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 10 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 11 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 12 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 13 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 14 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 15 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 16 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 17 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 18 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 19 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 20 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 21 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 22 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 23 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 24 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 25 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 26 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 27 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 28 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 29 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 30 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 31 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 32 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 33 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 34 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 35 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 36 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 37 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 38 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 39 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 40 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 41 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 42 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 43 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 44 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 45 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 46 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 47 Leader: 3 Replicas: 3 Isr: 3 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 48 Leader: 1 Replicas: 1 Isr: 1 Elr: N/A LastKnownElr: N/A

Topic: __consumer_offsets Partition: 49 Leader: 2 Replicas: 2 Isr: 2 Elr: N/A LastKnownElr: N/A

[root@k3 ~]# /opt/kafka/kafka_2.13-3.8.0/bin/kafka-topics.sh --describe --bootstrap-server 192.168.207.131:9092,192.168.207.132:9092,192.168.207.133:9092 --topic iom-int-topic

[2024-10-11 11:17:17,201] WARN [AdminClient clientId=adminclient-1] The DescribeTopicPartitions API is not supported, using Metadata API to describe topics. (org.apache.kafka.clients.admin.KafkaAdminClient)

Topic: iom-int-topic TopicId: oar-5HxnQSaOBbYX48rotw PartitionCount: 3 ReplicationFactor: 3 Configs:

Topic: iom-int-topic Partition: 0 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3 Elr: N/A LastKnownElr: N/A

Topic: iom-int-topic Partition: 1 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Elr: N/A LastKnownElr: N/A

Topic: iom-int-topic Partition: 2 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2 Elr: N/A LastKnownElr: N/A

[root@k3 ~]# grep factor /opt/kafka/kafka_2.13-3.8.0/config/kraft/server.properties

# The replication factor for the group metadata internal topics "__consumer_offsets" and "__transaction_state"

offsets.topic.replication.factor=3

transaction.state.log.replication.factor=3

[root@k3 ~]# grep factor /opt/kafka/kafka_2.13-3.8.0/config/server.properties

# The replication factor for the group metadata internal topics "__consumer_offsets" and "__transaction_state"

offsets.topic.replication.factor=3

transaction.state.log.replication.factor=3

[root@k3 ~]#

使用alter无法修改

[root@k3 ~]# /opt/kafka/kafka_2.13-3.8.0/bin/kafka-topics.sh --alter --topic __consumer_offsets --bootstrap-server 192.168.207.131:9092,192.168.207.132:9092,192.168.207.133:9092 --replication-factor 3 --partitions 50

Option "[replication-factor]" can't be used with option "[alter]"

Option Description

------ -----------

--alter Alter the number of partitions and

replica assignment. Update the

configuration of an existing topic

via --alter is no longer supported

here (the kafka-configs CLI supports

altering topic configs with a --

bootstrap-server option).

--at-min-isr-partitions if set when describing topics, only

show partitions whose isr count is

equal to the configured minimum.

--bootstrap-server <String: server to REQUIRED: The Kafka server to connect

正确的修改方法：

创建offset.json文件：

[root@k3 ~]# cat offset.json

{

"topics": [

{

"topic": "__consumer_offsets"

}

"version": 1

}

根据offset.json生成分区副本分配策略json文件：

/opt/kafka/kafka_2.13-3.8.0/bin/kafka-reassign-partitions.sh --bootstrap-server 192.168.207.131:9092,192.168.207.132:9092,192.168.207.133:9092 --topics-to-move-json-file offset.json --broker-list "1,2,3" –generate

修改replicas和log_dirs并保存上述输出的内容为off.json

[root@k3 ~]# cat off.json

执行修改操作：

[root@k3 ~]# /opt/kafka/kafka_2.13-3.8.0/bin/kafka-reassign-partitions.sh --bootstrap-server 192.168.207.131:9092,192.168.207.132:9092,192.168.207.133:9092 --reassignment-json-file off.json --execute

[2024-10-11 12:33:33,956] WARN [AdminClient clientId=reassign-partitions-tool] The DescribeTopicPartitions API is not supported, using Metadata API to describe topics. (org.apache.kafka.clients.admin.KafkaAdminClient)

Current partition replica assignment

Save this to use as the --reassignment-json-file option during rollback

Successfully started partition reassignments for __consumer_offsets-0,__consumer_offsets-1,__consumer_offsets-2,__consumer_offsets-3,__consumer_offsets-4,__consumer_offsets-5,__consumer_offsets-6,__consumer_offsets-7,__consumer_offsets-8,__consumer_offsets-9,__consumer_offsets-10,__consumer_offsets-11,__consumer_offsets-12,__consumer_offsets-13,__consumer_offsets-14,__consumer_offsets-15,__consumer_offsets-16,__consumer_offsets-17,__consumer_offsets-18,__consumer_offsets-19,__consumer_offsets-20,__consumer_offsets-21,__consumer_offsets-22,__consumer_offsets-23,__consumer_offsets-24,__consumer_offsets-25,__consumer_offsets-26,__consumer_offsets-27,__consumer_offsets-28,__consumer_offsets-29,__consumer_offsets-30,__consumer_offsets-31,__consumer_offsets-32,__consumer_offsets-33,__consumer_offsets-34,__consumer_offsets-35,__consumer_offsets-36,__consumer_offsets-37,__consumer_offsets-38,__consumer_offsets-39,__consumer_offsets-40,__consumer_offsets-41,__consumer_offsets-42,__consumer_offsets-43,__consumer_offsets-44,__consumer_offsets-45,__consumer_offsets-46,__consumer_offsets-47,__consumer_offsets-48,__consumer_offsets-49

再次检查：ReplicationFactor变成了3

/opt/kafka/kafka_2.13-3.8.0/bin/kafka-topics.sh --describe --bootstrap-server 192.168.207.131:9092,192.168.207.132:9092,192.168.207.133:9092 --topic iom-int-topic