1.ceph安装
1.1 环境准备
配置hosts：
$ vim /etc/hosts
10.121.x.x node01
设置ssh无密码登录：
# ssh-keygen -t rsa
# ssh-copy-id -i /root/.ssh/id_rsa node01
关闭selinux、firewalld
# setenforce 0
# sed -i "s#SELINUX=enforcing#SELINUX=disabled#g" /etc/selinux/config
# systemctl stop firewalld
# systemctl disable firewalld
配置ceph源：
# cd /etc/yum.repos.d/
# vim ceph.repo
[noarch]
name=noarch
baseurl=https://mirrors.aliyun.com/ceph/rpm-nautilus/el7/noarch/
enabled=1
gpgcheck=0
[x86_64]
name=x86 64
baseurl=https://mirrors.aliyun.com/ceph/rpm-nautilus/el7/x86_64/
enabled=1
gpgcheck=0
1.2 ceph 安装
本次安装使用ceph-deploy工具：
# yum -y install python-setuptools ceph-deploy
创建ceph操作目录，用于保存ceph-deploy生成的配置文件和密匙信息：
# mkdir /root/ceph-deploy
# cd ceph-deploy/
创建ceph集群：
# ceph-deploy new node01
手动安装软件包：
# yum install -y ceph ceph-mon ceph-mgr ceph-radosgw ceph-mds
monitor初始化：
# ceph-deploy mon create-initial
# ls -l
total 44
-rw------- 1 root root 113 Aug 21 18:39 ceph.bootstrap-mds.keyring
-rw------- 1 root root 113 Aug 21 18:39 ceph.bootstrap-mgr.keyring
-rw------- 1 root root 113 Aug 21 18:39 ceph.bootstrap-osd.keyring
-rw------- 1 root root 113 Aug 21 18:39 ceph.bootstrap-rgw.keyring
-rw------- 1 root root 151 Aug 21 18:39 ceph.client.admin.keyring
-rw-r--r-- 1 root root 198 Aug 21 18:37 ceph.conf
-rw-r--r-- 1 root root 16040 Aug 21 18:39 ceph-deploy-ceph.log
-rw------- 1 root root 73 Aug 21 18:37 ceph.mon.keyring
推送配置文件到节点：
# ceph-deploy admin node01
第二次推送需要加参数：
# ceph-deploy --overwrite-conf admin node01
创建manager daemon来做监控：
# ceph-deploy mgr create node01
# ceph -s
cluster:
id: cc039d05-2643-4967-a89c-39fa7cdfa695
health: HEALTH_WARN
OSD count 0 < osd_pool_default_size 3
mon is allowing insecure global_id reclaim
services:
mon: 1 daemons, quorum node01 (age 2m)
mgr: node01(active, since 26s)
osd: 0 osds: 0 up, 0 in
data:
pools: 0 pools, 0 pgs
objects: 0 objects, 0 B
usage: 0 B used, 0 B / 0 B avail
pgs:
添加osd：
# lsblk -l
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdd 8:48 0 3.7T 0 disk
sdb 8:16 0 3.7T 0 disk
sr0 11:0 1 1024M 0 rom
sde 8:64 0 893.3G 0 disk
sde2 8:66 0 1G 0 part /boot
sde3 8:67 0 892.3G 0 part
centos00-root 253:1 0 888.3G 0 lvm /
centos00-swap 253:0 0 4G 0 lvm
sde1 8:65 0 2M 0 part
sdc 8:32 0 3.7T 0 disk
# ceph-deploy osd create node01 --data /dev/sdb
# ceph-deploy osd create node01 --data /dev/sdc
# ceph-deploy osd create node01 --data /dev/sdd
# ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 10.91606 root default
-3 10.91606 host node01
0 hdd 3.63869 osd.0 up 1.00000 1.00000
1 hdd 3.63869 osd.1 up 1.00000 1.00000
2 hdd 3.63869 osd.2 up 1.00000 1.00000
修改ceph.conf添加如下参数：
# vim ceph.conf
osd pool default min_size = 1 #设置osd池的默认最小大小为1
osd pool default size = 1 # 设置osd池的默认大小为1
public_network = 10.121.218.0/24
mon_allow_pool_delete = true
# ceph-deploy --overwrite-conf admin node01
# ceph -s
cluster:
id: cc039d05-2643-4967-a89c-39fa7cdfa695
health: HEALTH_WARN
mon is allowing insecure global_id reclaim
services:
mon: 1 daemons, quorum node01 (age 4m)
mgr: node01(active, since 3m)
osd: 3 osds: 3 up (since 93s), 3 in (since 93s)
data:
pools: 0 pools, 0 pgs
objects: 0 objects, 0 B
usage: 3.0 GiB used, 11 TiB / 11 TiB avail
pgs:
说明：
查阅资料解决版本是禁用不安全模式：
# ceph config set mon auth_allow_insecure_global_id_reclaim false
1.3 创建池：
# ceph osd pool create cloudpods 64 64
# rbd pool init cloudpods
告警：
这里提示副本数没有配置告警
ceph -s
cluster:
id: cc039d05-2643-4967-a89c-39fa7cdfa695
health: HEALTH_WARN
1 pool(s) have no replicas configured
services:
mon: 1 daemons, quorum node01 (age 3m)
mgr: node01(active, since 3m)
osd: 3 osds: 3 up (since 3m), 3 in (since 10m)
data:
pools: 1 pools, 64 pgs
objects: 1 objects, 19 B
usage: 3.0 GiB used, 11 TiB / 11 TiB avail
pgs: 64 active+clean
屏蔽告警方法：
ceph config set global mon_warn_on_pool_no_redundancy false
重启服务器
2.cloudpods配置ceph
存储-块存储-新建：
然后关联一下宿主机，否则会显示离线：
验证：
创建虚拟机并选择ceph RBD
