安装Java
sudo apt install default-jdk
# 执行完直接直接查看版本就好了
java -version
https://blog.csdn.net/CyberSparkZ/article/details/132441191
安装zookeeper
https://blog.csdn.net/supercrsky/article/details/124570611
https://blog.csdn.net/xiaozhang_man/article/details/127178554
tar zxvf apache-zookeeper-3.8.0-bin.tar.gz -C /opt/
mv /opt/apache-zookeeper-3.8.0-bin/ /opt/zookeeper
cd /opt/zookeeper/conf/
cp zoo_sample.cfg zoo.cfg
vi zoo.cfg
# 修改如下内容
dataDir=/opt/zookeeper/zkData
dataLogDir=/opt/zookeeper/zkLog
cd /opt/zookeeper/bin/
# 启动zookeeper
./zkServer.sh start
# 查看进程是否启动
jps
# 查看状态
./zkServer.sh status
# 停止zookeeper
./zkServer.sh stop
安装kafka
tar -xzf kafka_2.13-3.1.0.tgz
如果使用集群,还要修改下面配置
https://www.bilibili.com/video/BV1vr4y1677k?p=6&vd_source=786c72dec46738f552f75a0a3abab675
启动命令:
# 先启动zookeeper
cd /opt/zookeeper/bin/
./zkServer.sh start
# 再启动kafka
cd /home/kafka_2.13-3.1.0/
bin/kafka-server-start.sh config/server.properties
此方式可以实时查看日志.
后台启动方式:
./kafka-server-start.sh -daemon ../config/server.properties
查询进程和关闭命令
jps
./kafka-server-stop.sh
kafka常见命令
#创建主题 主题名是 quickstart-events
$ bin/kafka-topics.sh --create --topic quickstart-events --bootstrap-server localhost:9092
#查询主题
$ bin/kafka-topics.sh --describe --topic quickstart-events --bootstrap-server localhost:9092
#主题中写入消息
bin/kafka-console-producer.sh --topic quickstart-events --bootstrap-server localhost:9092
This is my first event
This is my second event
#主题中读取消息
bin/kafka-console-consumer.sh --topic quickstart-events --from-beginning --bootstrap-server localhost:9092
This is my first event
This is my second event
flink中创建生产者往kafka发送数据
创建对应topic的Kafka消费者,运行程序,kafka消费者就会接收到生产者发送过来的消息
flink中创建消费中处理kafka中的数据
运行程序 ,启动一个消费者,然后运行上一个程序,启动生产者,就会获取到数据。