文章目录
- 简介
- 用法
简介
flume-ng-sql-source 作者已经停止维护了,并且已经不支持新版Flume,我们决定开启这一项目 flume-ng-sql。
用法
-
像 flume-ng-sql-source 一样将 flume-ng-sql 发行的 Jar 包导入
lib
下,必要时需要添加自己 MySQL 版本对应的驱动。
-
编写配置文件:
a1.sources = r1 a1.channels = c1 a1.sinks = k1 # 这是 flume-ng-sql 有(也是仅有)的一些配置 a1.sources.r1.type = ink.awning.flume.sql.SQLSource a1.sources.r1.sql.url = jdbc:mysql://localhost:3306/school a1.sources.r1.sql.username = root a1.sources.r1.sql.password = 1246 a1.sources.r1.sql.query = SELECT * FROM student a1.sources.r1.sql.delay = 1000 a1.sinks.k1.type = logger a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100 a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1 a1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink a1.sinks.k1.kafka.topic = 123 a1.sinks.k1.kafka.bootstrap.servers = localhost:9092 a1.sinks.k1.kafka.flumeBatchSize = 20 a1.sinks.k1.kafka.producer.acks = 1 a1.sinks.k1.kafka.producer.linger.ms = 1
-
启动 Kafka 及其消费者,启动 Flume(详细操作请参考其他文章),MySQL 中的数据便会传递给消费者: