进入docker中的logstash 容器内:
jelex@bogon ~ % docker exec -it 7ee8960c99a31e607f346b2802419b8b819cc860863bc283cb7483bc03ba1420 /bin/sh
$ pwd
/usr/share/logstash
$ ls
bin CONTRIBUTORS Gemfile jdk logstash-core modules tools x-pack
config data Gemfile.lock lib logstash-core-plugin-api pipeline vendor
$ cd bin
$ ls
benchmark.bat ingest-convert.bat logstash-keystore logstash-plugin.bat pqrepair.bat
benchmark.sh ingest-convert.sh logstash-keystore.bat pqcheck ruby
cpdump logstash logstash.lib.sh pqcheck.bat setup.bat
dependencies-report logstash.bat logstash-plugin pqrepair system-install
安装插件:
$ logstash-plugin install logstash-codec-json_lines
logstash优化:官方的建议是提高每次批处理的数量,调节传输间歇时间。当batch.size增大,es处理的事件数就会变少,写入也就越快了
vim /etc/logstash/pipeline.yml
pipeline线程数,官方建议是等于CPU内核数
pipeline.workers: 24
实际output时的线程数
pipeline.output.workers: 24
每次发送的事件数
pipeline.batch.size: 10000
发送延时(ms)
pipeline.batch.delay: 10