Beats 收集数据
Beats 是一个免费且开放的平台，集合了多种单一用途数据采集器。它们从成百上千或成千上万台机器 和系统向 Logstash 或 Elasticsearch 发送数据。
虽然利用 logstash 就可以收集日志，功能强大，但由于 Logtash 是基于Java实现，需要在采集日志的主 机上安装JAVA环境
logstash运行时最少也会需要额外的500M的以上的内存，会消耗比较多的内存和磁盘空间
可以采有基于Go开发的 Beat 工具代替 Logstash 收集日志，部署更为方便，而且只占用10M左右的内 存空间及更小的磁盘空间。
利用 Metricbeat 监控性能相关指标
Metricbeat 可以收集指标数据，比如系统运行状态、CPU、内存利用率等。
生产中一般用 zabbix 等专门的监控系统实现此功能
官方配置说明
https://www.elastic.co/guide/en/beats/metricbeat/current/configuring-howtometricbeat.html
https://www.elastic.co/guide/en/beats/metricbeat/current/configuring-howtometricbeat.html
下载 metricbeat 并安装
下载链接
https://www.elastic.co/cn/downloads/beats/metricbeat
https://mirrors.tuna.tsinghua.edu.cn/elasticstack/8.x/apt/pool/main/m/metricbeat/
https://mirrors.tuna.tsinghua.edu.cn/elasticstack/7.x/apt/pool/main/m/metricbeat/范例:
#新版下载
[root@elk-web2 ~]#wget
https://mirrors.tuna.tsinghua.edu.cn/elasticstack/8.x/apt/pool/main/m/metricbeat/metricbeat-8.6.1-amd64.deb
#旧版下载
[root@elk-web2 ~]#wget
https://mirrors.tuna.tsinghua.edu.cn/elasticstack/7.x/apt/pool/main/m/metricbeat/metricbeat-7.6.2-amd64.deb
[root@elk-web2 ~]#dpkg -i metricbeat-7.6.2-amd64.deb修改配置
[root@elk-web2 ~]#vim /etc/metricbeat/metricbeat.yml
#setup.kibana:
# host: "10.0.0.101:5601" #指向kabana服务器地址和端口，非必须项，即使不设置Kibana也可以通过ES获取Metrics信息
#-------------------------- Elasticsearch output -----------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["10.0.0.101:9200","10.0.0.102:9200","10.0.0.103:9200"] #指向任意一个ELK集群节点即可
[root@kibana ~]#grep -Ev "#|^$" /etc/metricbeat/metricbeat.yml
metricbeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 1
index.codec: best_compression
setup.kibana:
host: "localhost:5601"
output.elasticsearch:
hosts: ["10.0.0.181:9200","10.0.0.182:9200","10.0.0.183:9200"]
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
启动服务
[root@kibana ~]#systemctl enable --now metricbeat.service
[root@kibana ~]#systemctl status metricbeat.service
通过 Kibana 查看收集的性能指标
8.X版本界面
运行时间 --- invertory 库存
[Stack Management] ------- 数据视图
Discover
利用 Heartbeat 监控
heartbeat 用来定时探测服务是否正常运行。支持ICMP、TCP 和 HTTP，也支持TLS、身份验证和代理
官方heartbeat配置文档
https://www.elastic.co/guide/en/beats/heartbeat/current/configuring-howtoheartbeat.htmlhttps://www.elastic.co/guide/en/beats/heartbeat/current/configuring-howtoheartbeat.html
下载并安装
下载链接
https://www.elastic.co/cn/downloads/beats/heartbeat
https://mirrors.tuna.tsinghua.edu.cn/elasticstack/8.x/apt/pool/main/h/heartbeat-elastic/
https://mirrors.tuna.tsinghua.edu.cn/elasticstack/7.x/apt/pool/main/h/heartbeat-elastic/
#新版
[root@elk-web2 ~]#wget https://mirrors.tuna.tsinghua.edu.cn/elasticstack/8.x/apt/pool/main/h/heartbeat-elastic/heartbeat-8.6.1-amd64.deb
#旧版
[root@elk-web2 ~]#wget https://mirrors.tuna.tsinghua.edu.cn/elasticstack/7.x/apt/pool/main/h/heartbeat-elastic/heartbeat-7.6.2-amd64.deb
#安装
[root@elk-web2 ~]#dpkg -i heartbeat-7.6.2-amd64.deb
#准备需要监控的服务httpd
[root@elk-web2 ~]#apt -y install apache2修改配置
官方参考
https://www.elastic.co/guide/en/beats/heartbeat/current/configuration-heartbeat-options.html
#官方示例 heartbeat.yml
heartbeat.monitors:
- type: icmp
id: ping-myhost
name: My Host Ping
hosts: ["myhost"]
schedule: '*/5 * * * * * *' #相当于'@every 5s'
- type: tcp
id: myhost-tcp-echo
name: My Host TCP Echo
hosts: ["myhost:777"] # default TCP Echo Protocol
check.send: "Check"
check.receive: "Check"
schedule: '@every 5s'
- type: http
id: service-status
name: Service Status
service.name: my-apm-service-name
hosts: ["http://localhost:80/service/status"]
check.response.status: [200]
schedule: '@every 5s'
heartbeat.scheduler:
limit: 10
时间格式: 注意可以支持秒级精度
范例
[root@kibana ~]#vim /etc/heartbeat/heartbeat.yml
# Configure monitors inline
heartbeat.monitors:
- type: http
enabled: true #修改此行false为true
# List or urls to query
urls: ["http://localhost:80/test.html"] #修改此行，指向需要监控的服务器的地址和端口
# Configure task schedule
schedule: '@every 10s'
# Total test connection and data exchange timeout
timeout: 6s
- type: tcp
id: myhost-tcp-echo
name: My Host TCP Echo
hosts: ["10.0.0.205:80"] # default TCP Echo Protocol
schedule: '@every 5s'
- type: icmp #添加下面5行，用于监控ICMP
id: ping-myhost
name: My Host Ping
hosts: ["10.0.0.100"]
schedule: '*/5 * * * * * *'
setup.kibana:
# Kibana Host
# Scheme and port can be left out and will be set to the default (http and 5601)
# In case you specify and additional path, the scheme is required: http://localhost:5601/path
# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
host: "10.0.0.101:5601 #指向kibana服务器地址和端口
#-------------------------- Elasticsearch output ----------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["10.0.0.181:9200","10.0.0.182:9200","10.0.0.183:9200"] #修改此行，指向ELK集群服务器地址和端口
启动服务
[root@kibana ~]#systemctl enable --now heartbeat-elastic.service
head 插件查看索引
通过 Kibana 查看收集的性能指标
通过Kibana 可以看到主机的状态
新版
运行时间 --- 监测
