文章目录
- 一:前置准备-(参考之前博客):
- 1.1:准备Elasticsearch和Kibana环境:
- 1.1.1:地址:https://blog.csdn.net/Abraxs/article/details/128517777
- 二:Docker安装LogStash(数据收集引擎,具有实时管道功能):
- 2.1:拉取logstash镜像:
- 2.2:根据logstash镜像生成容器:
- 2.3:查看docker所有容器:
- 三:开放防火墙LogStash对应的端口:
- 四:修改LogStash配置[logstash.(yml / config)]:
- 4.1:进入容器修改logstash.yml配置文件:
- 4.1.1:也可以尝试通过挂在目录在宿主机配置文件操作
- 4.1.2:修改默认配置>:改成如下配置
- 4.2:进入容器修改pipeline文件夹下的logstash.conf配置文件:
- 4.2.1:配置文件默认信息:
- 4.2.2:修改填充默认配置文件信息:
- 4.3:重启logstash容器[docker restart containerid]
一:前置准备-(参考之前博客):
1.1:准备Elasticsearch和Kibana环境:
1.1.1:地址:https://blog.csdn.net/Abraxs/article/details/128517777
二:Docker安装LogStash(数据收集引擎,具有实时管道功能):
2.1:拉取logstash镜像:
[root@vboxnode3ccccccttttttchenyang ~]# docker pull logstash:6.6.0
6.6.0: Pulling from library/logstash
a02a4930cb5d: Pull complete
83cf3eaa08e1: Pull complete
162671d94cae: Pull complete
c678994d9b6f: Pull complete
af6d15336df1: Pull complete
b808ea4c2d1f: Pull complete
dc25014ab499: Pull complete
16159f779f38: Pull complete
521d45965a24: Pull complete
30fa9d097a91: Pull complete
066458677680: Pull complete
Digest: sha256:08bc3d552c6ec2d47e3970e063dfe800e3467ba1ef80fb87f37902daba9a560b
Status: Downloaded newer image for logstash:6.6.0
docker.io/library/logstash:6.6.0
[root@vboxnode3ccccccttttttcyang ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
wurstmeister/kafka latest 2dd91ce2efe1 18 months ago 508MB
influxdb 2.0 33f568b26cd9 19 months ago 342MB
grafana/grafana latest 9b957e098315 19 months ago 275MB
logstash 6.6.0 8f45a7702366 4 years ago 751MB
wurstmeister/zookeeper latest 3f43f72cb283 4 years ago 510MB
2.2:根据logstash镜像生成容器:
命令如下
docker run -d -p 9600:9600 -p 4560:4560 --name logstash logstash:6.6.0
[root@vboxnode3ccccccttttttcyang ~]# docker run -d -p 9600:9600 -p 4560:4560 --name logstash logstash:6.6.0
ae0fbdac223afd98a6b00cfb30f312d58217b725342848ad59370d9e5f7a18db
2.3:查看docker所有容器:
[root ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ae0fbdac223a logstash:6.6.0 "/usr/local/bin/dock…" 3 seconds ago Up 1 second 0.0.0.0:4560->4560/tcp, :::4560->4560/tcp, 0.0.0.0:9600->9600/tcp, :::9600->9600/tcp, 5044/tcp logstash
29753343c1b0 wurstmeister/zookeeper "/bin/sh -c '/usr/sb…" 2 months ago Up About an hour 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, :::2181->2181/tcp
三:开放防火墙LogStash对应的端口:
[root ~]# firewall-cmd --permanent -zone=public --add-port=9600/tcp
usage: see firewall-cmd man page
firewall-cmd: error: unrecognized arguments: -zone=public
[root ~]# firewall-cmd --permanent --zone=public --add-port=9600/tcp
success
[root ~]# firewall-cmd --permanent --zone=public --add-port=4560/tcp
success
[root ~]# firewall-cmd --reload
success
[root ~]# firewall-cmd --permanent --zone=public --list-port
3306/tcp 8848/tcp 8091/tcp 8879/tcp 8887/tcp 8083/tcp 8086/tcp 8090/tcp 8099/tcp 9600/tcp 4560/tcp
四:修改LogStash配置[logstash.(yml / config)]:
4.1:进入容器修改logstash.yml配置文件:
4.1.1:也可以尝试通过挂在目录在宿主机配置文件操作
docker exec -it ae0fbdac223a /bin/sh
[root ~]# docker exec -it ae0fbdac223a /bin/sh
sh-4.2$ ls
bin CONTRIBUTORS Gemfile lib logstash-core modules pipeline vendor
config data Gemfile.lock LICENSE.txt logstash-core-plugin-api NOTICE.TXT tools x-pack
sh-4.2$ cd config/
sh-4.2$ ls
jvm.options log4j2.properties logstash-sample.conf logstash.yml pipelines.yml startup.options
sh-4.2$
sh-4.2$ vi logstash.yml
配置文件默认信息:
4.1.2:修改默认配置>:改成如下配置
http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.url: http://192.yourIp.103:9200
# 如果有密码执行下面
xpack.monitoring.elasticsearch.username: elastic
xpack.monitoring.elasticsearch.password: 123456
4.2:进入容器修改pipeline文件夹下的logstash.conf配置文件:
[root@vboxnode3ccccccttttttchenyang ~]# docker exec -it ae0fbdac223a /bin/sh
sh-4.2$ ls
bin CONTRIBUTORS Gemfile lib logstash-core modules pipeline vendor
config data Gemfile.lock LICENSE.txt logstash-core-plugin-api NOTICE.TXT tools x-pack
sh-4.2$ cd config/
sh-4.2$ ls
jvm.options log4j2.properties logstash-sample.conf logstash.yml pipelines.yml startup.options
sh-4.2$ cd ..
sh-4.2$ ls
bin CONTRIBUTORS Gemfile lib logstash-core modules pipeline vendor
config data Gemfile.lock LICENSE.txt logstash-core-plugin-api NOTICE.TXT tools x-pack
sh-4.2$ cd pipeline/
sh-4.2$ ls
logstash.conf
sh-4.2$ vi logstash.conf
sh-4.2$
sh-4.2$ pwd
/usr/share/logstash/pipeline
4.2.1:配置文件默认信息:
input {
beats {
port => 5044
}
}
output {
stdout {
codec => rubydebug
}
}
4.2.2:修改填充默认配置文件信息:
input {
tcp {
mode => "server"
#logstash配置地址
host => "0.0.0.0"
port => 4560
#日志文件json输出
codec => json_lines
}
}
output {
elasticsearch {
action => "index"
# ES服务地址,也可以是数组多地址
hosts => ["192.168.56.102:9200"]
# Es index > 类似DB数据库
index => "index-logstash"
}
stdout {
codec => rubydebug
}
}