文章目录
- 1.etcd基本概念
- 2.etcd的基本知识
- 3.etcd优势
- 4.etcd单机部署
- 4.1 linux部署
- 4.2 windows部署
- 4.3 docker安装etcd
- 5.etcd集群部署
1.etcd基本概念
etcd是一个高可用的分布式键值存储系统,是CoreOS(现在隶属于Red Hat)公司开发的一个开源项目。它提供了一个简单的接口来存储和检索键值对数据,并使用Raft协议实现了分布式一致性。etcd广泛应用于Docker、Kubernetes等分布式系统中,用于存储配置信息、服务发现、领导者选举等方面。
2.etcd的基本知识
- Etcd的定义:Etcd是一个分布式的、高可用的键值存储系统,具有快速响应、支持高并发等特点。
- Etcd的优点:高可用、数据一致性、快速响应、支持分布式事务。
- Etcd的应用场景:存储集群中各种配置信息、元数据以及服务发现等。
- Etcd的数据结构:类似于键值对的数据结构,支持多种数据类型,如字符串、整数、布尔、数组等。
- Etcd的API:提供丰富的API接口,通过HTTP协议进行访问和操作。
- Etcd的数据同步:Etcd使用Raft协议来保证数据的一致性和高可用性,在多个节点之间同步数据。
- Etcd的集群部署:Etcd可以部署为单节点或多节点集群,多节点集群可以提高系统的可用性和性能。
- Etcd的安全性:Etcd提供了诸如SSL/TLS加密、授权、访问控制等安全机制,保障了数据的安全性。
- Etcd的监控:Etcd提供了各种监控指标和工具,方便管理员对集群进行监控和管理。
- Etcd的常用工具:包括etcdctl命令行工具、etcd browser浏览器、etcd dashboard仪表板、etcdwatch等。
3.etcd优势
etcd是一种分布式键值存储系统,具有以下特点和优势:
- 强一致性:etcd使用Raft协议实现强一致性,确保每个节点上的数据始终保持一致。
- 高可用性:etcd可以通过多副本和自动故障转移机制来实现高可用性,即使某个节点故障,也可以保证服务不中断。
- 高性能:etcd采用预写日志(WAL)技术,可以快速写入数据,同时还支持快速读取和查询。
- 分布式:etcd可以在多台服务器上运行,数据可以在多个节点之间分布,提高了系统的可伸缩性和性能。
- 安全性:etcd支持TLS加密和认证,确保数据传输的安全性。
- 简单易用:etcd提供简单易用的HTTP/JSON API,可以方便地进行数据读写和查询。
- 开源免费:etcd是一个完全开源的项目,具有Apache 2.0许可证。
4.etcd单机部署
4.1 linux部署
下载地址:https://github.com/etcd-io/etcd/releases/download/v3.5.13/etcd-v3.5.13-linux-amd64.tar.gz
上传服务器:
tar xf etcd-v3.5.13-linux-amd64.tar.gz
mkdir -p /opt/etcd/bin
cp -a etcd-v3.5.13-linux-amd64/etcd* /opt/etcd/bin/
cd /opt/etcd/
mkdir -p data conf
创建配置文件
cat << 'EOF' > /opt/etcd/conf/etcd.conf
ETCD_NAME="apisix"
ETCD_DATA_DIR="/opt/etcd/data"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379,http://127.0.0.1:2379"
ETCD_LISTEN_PEER_URLS="http://0.0.0.0:2380"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://0.0.0.0:2380,http://127.0.0.1:2380"
ETCD_ENABLE_V2="true"
EOF
注册成一个服务
cat << 'EOF' > /usr/lib/systemd/system/etcd.service
[Unit]
Description=etcd service
Documentation=https://github.com/etcd-io/etcd
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
User=root
Type=notify
EnvironmentFile=/opt/etcd/conf/etcd.conf
WorkingDirectory=/opt/etcd
ExecStart=/opt/etcd/bin/etcd
Restart=always
RestartSec=10s
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
启动服务
systemctl daemon-reload
systemctl enable --now etcd
netstat -ntplu | egrep etcd
tcp6 0 0 :::2379 :::* LISTEN 1228/etcd
tcp6 0 0 :::2380 :::* LISTEN 1228/etcd
设置环境变量
echo 'export PATH=$PATH:/opt/etcd/bin' > /etc/profile.d/etcd.sh
source /etc/profile.d/etcd.sh
查看状态
etcdctl endpoint status --write-out=table
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| 127.0.0.1:2379 | 17ac6c9b1b6e7f0c | 3.5.13 | 20 kB | true | false | 2 | 4 | 4 | |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
etcdctl endpoint health --write-out=table
+----------------+--------+------------+-------+
| ENDPOINT | HEALTH | TOOK | ERROR |
+----------------+--------+------------+-------+
| 127.0.0.1:2379 | true | 3.571276ms | |
+----------------+--------+------------+-------+
etcdctl member list --write-out=table
+------------------+---------+--------+-------------------------------------------+-------------------------------------------+------------+
| ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS | IS LEARNER |
+------------------+---------+--------+-------------------------------------------+-------------------------------------------+------------+
| 17ac6c9b1b6e7f0c | started | apisix | http://0.0.0.0:2380,http://127.0.0.1:2380 | http://0.0.0.0:2379,http://127.0.0.1:2379 | false |
+------------------+---------+--------+-------------------------------------------+-------------------------------------------+------------+
不变成服务, 直接启动etcd:
cd /opt/etcd/
./etcd
4.2 windows部署
安装地址:https://github.com/etcd-io/etcd/releases/tag/v3.5.14
4.3 docker安装etcd
mkdir -p /opt/soft/etcd/data
# 拉取etcd镜像
docker pull quay.io/coreos/etcd:v3.5.13
docker run -d \
-p 2379:2379 \
-p 2380:2380 \
--restart=always \
--mount type=bind,source=/opt/soft/etcd/data,destination=/etcd-data \
--name etcd \
quay.io/coreos/etcd:v3.5.13 \
/usr/local/bin/etcd \
--name s1 \
--data-dir /etcd-data \
--listen-client-urls http://0.0.0.0:2379 \
--advertise-client-urls http://0.0.0.0:2379 \
--listen-peer-urls http://0.0.0.0:2380 \
--initial-advertise-peer-urls http://0.0.0.0:2380 \
--initial-cluster s1=http://0.0.0.0:2380 \
--initial-cluster-token tkn \
--initial-cluster-state new \
--log-level info \
--logger zap \
--log-outputs stderr
# 查看Etcd服务器版本
docker exec etcd /usr/local/bin/etcd --version
# 查看Etcd客户端版本
docker exec etcd /usr/local/bin/etcdctl version
docker exec etcd /usr/local/bin/etcdutl version
# 查看Etcd健康状况
docker exec etcd /usr/local/bin/etcdctl endpoint health
# Etcd添加数据
docker exec etcd /usr/local/bin/etcdctl put foo bar
# Etcd查看数据
docker exec etcd /usr/local/bin/etcdctl get foo
# Etcd查看所有数据
docker exec etcd /usr/local/bin/etcdctl get --prefix ""
# Etcd删除数据
docker exec etcd /usr/local/bin/etcdctl del foo
[root@node1192 soft]# docker exec etcd /usr/local/bin/etcd --version
etcd Version: 3.5.13
Git SHA: c9063a0dc
Go Version: go1.21.8
Go OS/Arch: linux/amd64
[root@node1192 soft]# docker exec etcd /usr/local/bin/etcdctl version
etcdctl version: 3.5.13
API version: 3.5
[root@node1192 soft]# docker exec etcd /usr/local/bin/etcdutl version
etcdutl version: 3.5.13
API version: 3.5
[root@node1192 soft]# docker exec etcd /usr/local/bin/etcdctl endpoint health
127.0.0.1:2379 is healthy: successfully committed proposal: took = 5.561765ms
[root@node1192 soft]# docker exec etcd /usr/local/bin/etcdctl put foo bar
OK
[root@node1192 soft]# docker exec etcd /usr/local/bin/etcdctl get foo
foo
bar
[root@node1192 soft]# docker exec etcd /usr/local/bin/etcdctl get --prefix ""
foo
bar
[root@node1192 soft]# docker exec etcd /usr/local/bin/etcdctl del foo
1
5.etcd集群部署
#每台主机分别执行
hostnamectl set-hostname etcd-node01
hostnamectl set-hostname etcd-node02
hostnamectl set-hostname etcd-node03
#每台主机执行
cat << 'EOF' > /etc/hosts
192.168.66.101 etcd-node01
192.168.66.102 etcd-node02
192.168.66.103 etcd-node03
EOF
yum install ntpdate -y
ntpdate -s ntp1.aliyun.com
tar xf etcd-v3.5.13-linux-amd64.tar.gz
mkdir -pv /opt/etcd/{bin,data,conf}
cp -a etcd-v3.5.13-linux-amd64/etcd* /opt/etcd/bin/
编写配置文件
cat << 'EOF' > /opt/etcd/conf/etcd.conf
ETCD_NAME=etcd-node01
ETCD_DATA_DIR="/opt/etcd/data"
ETCD_LISTEN_PEER_URLS="http://192.168.66.101:2380,http://127.0.0.1:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.66.101:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.66.101:2380"
ETCD_INITIAL_CLUSTER="etcd-node01=http://192.168.66.101:2380,etcd-node02=http://192.168.66.102:2380,etcd-node03=http://192.168.66.103:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.66.101:2379"
EOF
cat << 'EOF' > /opt/etcd/conf/etcd.conf
ETCD_NAME=etcd-node02
ETCD_DATA_DIR="/opt/etcd/data"
ETCD_LISTEN_PEER_URLS="http://192.168.66.102:2380,http://127.0.0.1:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.66.102:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.66.102:2380"
ETCD_INITIAL_CLUSTER="etcd-node01=http://192.168.66.101:2380,etcd-node02=http://192.168.66.102:2380,etcd-node03=http://192.168.66.103:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.66.102:2379"
EOF
cat << 'EOF' > /opt/etcd/conf/etcd.conf
ETCD_NAME=etcd-node03
ETCD_DATA_DIR="/opt/etcd/data"
ETCD_LISTEN_PEER_URLS="http://192.168.66.103:2380,http://127.0.0.1:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.66.103:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.66.103:2380"
ETCD_INITIAL_CLUSTER="etcd-node01=http://192.168.66.101:2380,etcd-node02=http://192.168.66.102:2380,etcd-node03=http://192.168.66.103:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.66.103:2379"
EOF
注册服务
cat << 'EOF' > /usr/lib/systemd/system/etcd.service
[Unit]
Description=etcd service
Documentation=https://github.com/etcd-io/etcd
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
User=root
Type=notify
EnvironmentFile=/opt/etcd/conf/etcd.conf
WorkingDirectory=/opt/etcd
ExecStart=/opt/etcd/bin/etcd
Restart=always
RestartSec=10s
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
开启服务和环境变量
systemctl enable --now etcd
echo 'PATH=$PATH:/opt/etcd/bin' > /etc/profile.d/etcd.sh
source /etc/profile.d/etcd.sh
查看集群信息
etcdctl --endpoints=192.168.66.101:2379,192.168.66.102:2379,192.168.66.103:2379 endpoint status --write-out=table
+----------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| 192.168.66.101:2379 | 8dc9f79502ff52fe | 3.5.13 | 20 kB | false | false | 2 | 13 | 13 | |
| 192.168.66.102:2379 | 8fdf9dd965ff87d5 | 3.5.13 | 20 kB | true | false | 2 | 13 | 13 | |
| 192.168.66.103:2379 | fc380e2e4f60630e | 3.5.13 | 20 kB | false | false | 2 | 13 | 13 | |
+----------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
测试故障:将上面的 leader 节点关机,然后查看 leader 节点是否会在其他节点上启动。
etcdctl --endpoints=192.168.66.101:2379,192.168.66.102:2379,192.168.66.103:2379 endpoint status --write-out=table
+----------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| 192.168.66.101:2379 | 8dc9f79502ff52fe | 3.5.13 | 20 kB | true | false | 3 | 14 | 14 | |
| 192.168.66.103:2379 | fc380e2e4f60630e | 3.5.13 | 20 kB | false | false | 3 | 14 | 14 | |
+----------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
leader节点已经漂移到 192.168.66.101 上面了。
数据写入测试
在 etcd-node01 写入数据
root@etcd-node01(192.168.66.101)~>etcdctl put greeting "hello etcd"
OK
在 etcd-node03 读取数据
root@etcd-node03(192.168.66.103)~>etcdctl get greeting
greeting
hello etcd