1 前期规划
系统规划
本地磁盘规划
共享存储规划
DMDSC 集群为了实现多实例同时访问和修改数据，需要数据文件、控制文件和日志文件都放到共享存储上。DM 支持两种共享存储，裸设备和 DMASM，裸设备是未经过格式化的特殊字符设备，推荐使用 DMASM 共享存储。
其中，VOTE 是表决磁盘，一个集群一个表决磁盘。DCR 是 DM 集群注册表的简称，用于存储、维护集群配置的详细信息，一个集群一个注册表。生产环境中，共享存储一般由操作系统管理员进行划分。共享存储规划可参照下表：
端口规划
2 创建共享磁盘
两台虚拟机关机
1.1 DSC1创建以及添加共享磁盘
创建共享磁盘：(注意将虚拟机的安装目录添加到环境变量)
WIN+R，然后输入cmd进入命令行窗口：
E:\VM>vmware-vdiskmanager.exe -c -s 20GB -a lsilogic -t 2 "E:\vm1\DCS-share-20G.vmdk"
Creating disk 'E:\vm1\DCS-share-20G.vmdk'
Create: 100% done.
Virtual disk creation successful.
添加共享磁盘：
1.2 DSC2添加共享磁盘
虚拟磁盘设置完毕后，可以从设备属性检查相关设置是否生效：
当两台虚拟机添加完磁盘后先不要开机，虚拟机路径下的 .vmx 文件中会出现如下信息：两台都加
添加如下内容：
diskLib.dataCacheMaxSize=0
diskLib.dataCacheMaxReadAheadSize=0
diskLib.dataCacheMinReadAheadSize=0
diskLib.dataCachePageSize=4096
diskLib.maxUnsyncedWrites = “0”
disk.locking = “FALSE”
disk.locking必填，否则重启虚拟机操作系统时会提示磁盘被锁定
添加完成后，启动两台虚拟机
fdisk -l --可以查看共享磁盘信息
sd 代表的是 SCSI，SATA 接口硬盘
两台机器共享磁盘一致，共享磁盘添加成功。
DMDSC01
DMDSC02
3搭建DSC集群
3.1 安装数据库
参考：https://eco.dameng.com/document/dm/zh-cn/start/install-dm-linux-prepare.html
注意：关闭防火墙
[root@localhost /]# systemctl stop firewalld
3.2共享磁盘分区
执行以下命令查看磁盘
DMDSC01
[root@localhost ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 19G 0 part
├─centos-root 253:0 0 17G 0 lvm /
└─centos-swap 253:1 0 2G 0 lvm [SWAP]
sdb 8:16 0 20G 0 disk
sr0 11:0 1 4.4G 0 rom /run/media/cathy/CentOS 7 x86_64
[root@localhost ~]#
DMDSC02
[root@localhost ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 19G 0 part
├─centos-root 253:0 0 17G 0 lvm /
└─centos-swap 253:1 0 2G 0 lvm [SWAP]
sdb 8:16 0 20G 0 disk
sr0 11:0 1 4.4G 0 rom /run/media/cathy/CentOS 7 x86_64
[root@localhost ~]#
进行裸设备划分 （分区）–一个节点执行即可
因为是共享磁盘，只需在一台虚拟机操作即可
输入
fdisk /dev/sdb
依次输入 n → p → 1 →回车→回车→ +100M →回车，完成第一块磁盘划分
依次输入 n → p → 2 →回车→回车→ +100M →回车，完成第二块磁盘划分
依次输入 n → p → 3 →回车→回车→ +2048M →回车，完成第三块磁盘划分
依次输入 n → p → 4 →回车→回车→回车，完成第四块磁盘划分
w
保存配置并退出
[root@localhost ~]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xcdb0fcab.
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-41943039, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-41943039, default 41943039): +100M
Partition 1 of type Linux and of size 100 MiB is set
Command (m for help): n
Partition type:
p primary (1 primary, 0 extended, 3 free)
e extended
Select (default p): p
Partition number (2-4, default 2): 2
First sector (206848-41943039, default 206848):
Using default value 206848
Last sector, +sectors or +size{K,M,G} (206848-41943039, default 41943039): +100M
Partition 2 of type Linux and of size 100 MiB is set
Command (m for help): n
Partition type:
p primary (2 primary, 0 extended, 2 free)
e extended
Select (default p): p
Partition number (3,4, default 3): 3
First sector (411648-41943039, default 411648):
Using default value 411648
Last sector, +sectors or +size{K,M,G} (411648-41943039, default 41943039): +2048 M
Partition 3 of type Linux and of size 2 GiB is set
Command (m for help): n
Partition type:
p primary (3 primary, 0 extended, 1 free)
e extended
Select (default e): p
Selected partition 4
First sector (4605952-41943039, default 4605952):
Using default value 4605952
Last sector, +sectors or +size{K,M,G} (4605952-41943039, default 41943039):
Using default value 41943039
Partition 4 of type Linux and of size 17.8 GiB is set
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@localhost ~]#
查看配置结果：
[root@localhost ~]# fdisk -l
Disk /dev/sda: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000b12e7
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 2099199 1048576 83 Linux
/dev/sda2 2099200 41943039 19921920 8e Linux LVM
Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xcdb0fcab
Device Boot Start End Blocks Id System
/dev/sdb1 2048 206847 102400 83 Linux
/dev/sdb2 206848 411647 102400 83 Linux
/dev/sdb3 411648 4605951 2097152 83 Linux
/dev/sdb4 4605952 41943039 18668544 83 Linux
Disk /dev/mapper/centos-root: 18.2 GB, 18249416704 bytes, 35643392 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos-swap: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
[root@localhost ~]#
3.3 新建udev规则文件-裸设备绑定
（两台均需操作）
[root@localhost ~]# vim /etc/udev/rules.d/70-persistent-ipoib.rules
添加以下内容：
ACTION==“add”,KERNEL==“sdb1”,RUN+=“/bin/raw /dev/raw/raw1 %N”
ACTION==“add”,KERNEL==“sdb2”,RUN+=“/bin/raw /dev/raw/raw2 %N”
ACTION==“add”,KERNEL==“sdb3”,RUN+=“/bin/raw /dev/raw/raw3 %N”
ACTION==“add”,KERNEL==“sdb4”,RUN+=“/bin/raw /dev/raw/raw4 %N”
ACTION==“add”,KERNEL==“raw[1-4]”, OWNER=“dmdba”, GROUP=“dinstall”, MODE=“660”
执行以下命令使文件生效
[root@localhost ~]# udevadm trigger --type=devices --action=change
3.4 检查配置是否成功
[root@localhost ~]# ll /dev/raw/raw*
crw-rw----. 1 root disk 162, 0 Feb 19 00:54 /dev/raw/rawctl
[root@localhost ~]#
没有绑定成功，重启服务器，再验证绑定成功
[root@localhost ~]# reboot
DMDSC01 DMDSC02均显示：
[cathy@localhost ~]$ ll /dev/raw/raw*
crw-rw----. 1 dmdba dinstall 162, 1 Feb 19 00:56 /dev/raw/raw1
crw-rw----. 1 dmdba dinstall 162, 2 Feb 19 00:56 /dev/raw/raw2
crw-rw----. 1 dmdba dinstall 162, 3 Feb 19 00:56 /dev/raw/raw3
crw-rw----. 1 dmdba dinstall 162, 4 Feb 19 00:56 /dev/raw/raw4
crw-rw----. 1 root disk 162, 0 Feb 19 00:56 /dev/raw/rawctl
[cathy@localhost ~]$
注意：如果查不到配置的裸设备，请重启服务器在验证。
3.5 配置DCR初始化配置文件-创建配置文件存放目录
两个节点用dmdba用户执行。
mkdir -p /dm8/dmdbms/dsc_config
mkdir -p /dm8/dmdbms/dsc_config
3.6 新建dmdcr_cfg.ini文件
使用dmdba用户–两个节点都配置
全部使用外网或全部为内网ip均可以。（规范内网ip配置）
[dmdba@~]# vi /dm8/dmdbms/dsc_config/dmdcr_cfg.ini
DCR_N_GRP = 3 ##集群环境有多少个 GROUP，范围：1~16
DCR_VTD_PATH = /dev/raw/raw2 ##规划为 vote 的磁盘
DCR_OGUID = 210715 ## 消息标识，一个组里面只有一个。
[GRP] #新建一个 GROUP
DCR_GRP_TYPE = CSS ##组类型（CSS/ASM/DB）
DCR_GRP_NAME = GRP_CSS ##组名
DCR_GRP_N_EP = 2 ##组内节点个数
DCR_GRP_DSKCHK_CNT = 65 ##磁盘心跳容错时间，单位:秒
[GRP_CSS]
DCR_EP_NAME = CSS0 ##CSS 节点名
DCR_EP_HOST = 192.168.145.128 ##心跳地址
DCR_EP_PORT = 11286 ##CSS 端口
[GRP_CSS]
DCR_EP_NAME = CSS1
DCR_EP_HOST = 192.168.145.129
DCR_EP_PORT = 11287
[GRP]
DCR_GRP_TYPE = ASM
DCR_GRP_NAME = GRP_ASM
DCR_GRP_N_EP = 2
DCR_GRP_DSKCHK_CNT = 61
[GRP_ASM]
DCR_EP_NAME = ASM0 ##ASM 节点名，和 dmasvrmal 的 MAL_INST_NAME 一致
DCR_EP_SHM_KEY = 42424 ##共享内存标识
DCR_EP_SHM_SIZE = 1024 ##共享内存大小
DCR_EP_HOST = 192.168.145.128 ##心跳地址
DCR_EP_PORT = 11276 ##ASM 端口
DCR_EP_ASM_LOAD_PATH = /dev/raw
[GRP_ASM]
DCR_EP_NAME = ASM1
DCR_EP_SHM_KEY = 42425
DCR_EP_SHM_SIZE = 1024
DCR_EP_HOST = 192.168.145.129
DCR_EP_PORT = 11277
DCR_EP_ASM_LOAD_PATH = /dev/raw
[GRP]
DCR_GRP_TYPE = DB
DCR_GRP_NAME = GRP_DSC
DCR_GRP_N_EP = 2
DCR_GRP_DSKCHK_CNT = 57
[GRP_DSC]
DCR_EP_NAME = DSC0 ##实例名，和 dm.ini 的 INSTANCE_NAME 一致
DCR_EP_SEQNO = 0 ##组内序号，不能重复
DCR_EP_PORT = 5236 ##实例端口，和 dm.ini 的 PORT_NUM 一致
DCR_CHECK_PORT = 11256 ##DCR 检查端口
[GRP_DSC]
DCR_EP_NAME = DSC1
DCR_EP_SEQNO = 1
DCR_EP_PORT = 5237
DCR_CHECK_PORT = 11257
3.7创建ASM磁盘
使用dmdba用户创建ASM磁盘
[dmdba@localhost bin]$ ./dmasmcmd
dmasmcmd V8
ASM>create dcrdisk '/dev/raw/raw1' 'dcr'
[TRACE]The ASM initialize dcrdisk /dev/raw/raw1 to name DMASMdcr
Used time: 62.779(ms).
ASM>create votedisk '/dev/raw/raw2' 'vote'
[TRACE]The ASM initialize votedisk /dev/raw/raw2 to name DMASMvote
Used time: 6.580(ms).
ASM>create asmdisk '/dev/raw/raw3' 'LOG0'
[TRACE]The ASM initialize asmdisk /dev/raw/raw3 to name DMASMLOG0
Used time: 8.284(ms).
ASM>create asmdisk '/dev/raw/raw4' 'DATA0'
[TRACE]The ASM initialize asmdisk /dev/raw/raw4 to name DMASMDATA0
Used time: 15.007(ms).
ASM>
检查：
ASM>listdisks '/dev/raw/'
[/dev/raw//raw3]: Unused ASM disk, name:[DMASMLOG0], size:[2047M]
[/dev/raw//raw1]: Used ASM disk, name:[DMASMdcr], size:[99M], group_id:[0], disk_id:[0]
[/dev/raw//raw4]: Unused ASM disk, name:[DMASMDATA0], size:[18230M]
[/dev/raw//raw2]: Used ASM disk, name:[DMASMvote], size:[99M], group_id:[0], disk_id:[0]
Used time: 1.975(ms).
ASM>
3.8 使用dmdcr_dfg.ini配置文件初始化dcrdisk和votedisk
并在ASM提示符执行以下命令：
ASM>init dcrdisk '/dev/raw/raw1' from '/dm8/dmdbms/dsc_config/dmdcr_cfg.ini' identified by '123456'
[TRACE]DG 126 alloc extent for inode (0, 0, 1)
[TRACE]DG 126 alloc 4 extents for 0xfe000002 (0, 0, 2)->(0, 0, 5)
Used time: 77.403(ms).
ASM>init votedisk '/dev/raw/raw2' from '/dm8/dmdbms/dsc_config/dmdcr_cfg.ini'
[TRACE]DG 125 alloc extent for inode (0, 0, 1)
[TRACE]DG 125 alloc 4 extents for 0xfd000002 (0, 0, 2)->(0, 0, 5)
Used time: 58.036(ms).
ASM>
3.9 配置ASM的MAL系统配置文件
两个节点都需要配置，内容相同。
[dmdba@localhost ~]$ vim /dm8/dmdbms/dsc_config/dmasvrmal.ini
[MAL_INST0]
MAL_INST_NAME = ASM0
MAL_HOST = 192.168.145.128
MAL_PORT = 11266
[MAL_INST1]
MAL_INST_NAME = ASM1
MAL_HOST = 192.168.145.129
MAL_PORT = 11267
~
~
~
3.10 配置DCR启动配置文件
DSC01 机器配置 dmdcr.ini，dmdcr_seqo 为 0。
[dmdba@localhost ~]$ vim /dm8/dmdbms/dsc_config/dmdcr.ini
DMDCR_PATH = /dev/raw/raw1
DMDCR_MAL_PATH = /dm8/dmdbms/dsc_config/dmasvrmal.ini #dmasmsvr 使用的 MAL 配置文件路径
DMDCR_SEQNO = 0
##ASM 重启参数，命令行方式启动，注释掉的CMD为注册后修改的启动命令
DMDCR_ASM_RESTART_INTERVAL = 0 ##CSS 认定 ASM 故障重启的时间
DMDCR_ASM_STARTUP_CMD = /dm8/bin/dmasmsvr dcr_ini=/dm8/dmdbms/dsc_config/dmdcr.ini
##DMDCR_ASM_STARTUP_CMD = /dm8/dmdbms/dsc_config/DmAsmService_DSC0 start
##DB 重启参数，命令行方式启动
DMDCR_DB_RESTART_INTERVAL = 0 ##CSS 认定 DSC 故障重启的时间，设置为 0 不自动拉起
DMDCR_DB_STARTUP_CMD = /dm8/bin/dmserver path=/dm8/dmdbms/dsc_config/DSC0/dm.ini dcr_ini=/dm8/dmdbms/dsc_config/dmdcr.ini
##DMDCR_DB_STARTUP_CMD = /dm8/dmdbms/dsc_config/DmService_DSC0 start
~
DSC02 机器配置 dmdcr.ini，dmdcr_seqo 为 1。
[dmdba@localhost ~]$ vim /dm8/dmdbms/dsc_config/dmdcr.ini
CR_PATH = /dev/raw/raw1
DMDCR_MAL_PATH = /dm8/dmdbms/dsc_config/dmasvrmal.ini #dmasmsvr 使用的 MAL 配置文件路径
DMDCR_SEQNO =1
##ASM 重启参数，命令行方式启动，注释掉的CMD为注册后修改的启动命令
DMDCR_ASM_RESTART_INTERVAL = 0 ##CSS 认定 ASM 故障重启的时间
DMDCR_ASM_STARTUP_CMD = /dm8/bin/dmasmsvr dcr_ini= /dm8/dmdbms/dsc_config/dmdcr.ini
##DMDCR_ASM_STARTUP_CMD = /dm8/dmdbms/dsc_config/DmAsmService_DSC0 start
##DB 重启参数，命令行方式启动
DMDCR_DB_RESTART_INTERVAL = 0 ##CSS 认定 DSC 故障重启的时间，设置为 0 不自动拉起
DMDCR_DB_STARTUP_CMD = /dm8/bin/dmserver path=/dm8/dmdbms/dsc_config/DSC1/dm.ini dcr_ini= /dm8/dmdbms/dsc_config/dmdcr.ini
##DMDCR_DB_STARTUP_CMD = /dm8/dmdbms/dsc_config/DmService_DSC0 start
~
3.11 启动DMCSS服务
关闭防火墙
[root@localhost ~]# systemctl stop firewalld
[root@localhost ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@localhost ~]# ^C
关闭SELinux:
##临时关闭 SELinux：
[root@localhost ~]# setenforce 0
##临时开启 SELinux：
[root@localhost ~]# setenforce 1
##查看当前状态命令：
[root@localhost ~]# getenforce
Permissive
##永久关闭 SELinux 需修改配置文件：/etc/selinux/config，修改 SELINUX=disabled
[root@localhost ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled #修改为 disabled
# SELINUXTYPE= can take one of three two values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
两个节点启动dmcss，dmdba用户到数据库安装目录bin下执行
点一和二这里需要一起启动css只起一个节点会卡住不往下执行：
启动后会一直报错关于dm.ini的，不需要管，因为没初始化实例。后面初始化实例后就正常了
[dmdba@localhost ~]$ cd /dm8/bin
[dmdba@localhost bin]$ ./dmcss dcr_ini=/dm8/dmdbms/dsc_config/dmdcr.ini
DMCSS V8
DMCSS IS READY
[2024-02-20 22:08:35:137] [CSS]: Set EP CSS1[1] as Control node
3.12 启动DMASM 服务
两个节点均启动
[dmdba@localhost bin]$ /dm8/bin/dmasmsvr DCR_INI=/dm8/dmdbms/dsc_config/dmdcr.ini
ASM SELF EPNO:0
DMASMSVR V8
启动报错如下：dmasvrmal.ini节点名称配置错误
ASM SELF EPNO:1
DMASMSVR V8
Invalid [mal_name] or the file contains unrecognized characters in [/dm8/dmdbms/dsc_config/dmasvrmal.ini]!
Read ini file(/dm8/dmdbms/dsc_config/dmasvrmal.ini) error in line 3, code(-104)
mal cfg sys init error, code:[-104], desc:[Invalid INI file].
启动再次报错：
查看日志，发现两个节点的网络不畅通 cat /dm8/log/dmasm02_202402.log
2024-02-21 22:50:47.819 [ERROR] dmasmsvr P0000034610 T0000000000000034621 Can't connect to DM server on '192.168.145.129' port(11267) errno(101)
2024-02-21 22:50:47.819 [INFO] dmasmsvr P0000034610 T0000000000000034614 mal_site_port_get site_magic:27737, src_site:0, dst_site:0
2024-02-21 22:50:47.819 [INFO] dmasmsvr P0000034610 T0000000000000034620 site[0] mal_site_data_port_set from site[0, IP: 192.168.145.128, port_num: 11266], socket handle = 10, site_magic = 27737, link_seq = 1
2024-02-21 22:50:47.819 [INFO] dmasmsvr P0000034610 T0000000000000034615 mal_site_port_get site_magic:27737, src_site:0, dst_site:0
2024-02-21 22:52:52.805 [FATAL] dmasmsvr P0000034610 T0000000000000034610 sigterm_handler receive signal 2
2024-02-21 22:52:52.805 [FATAL] dmasmsvr P0000034610 T0000000000000034610 [for dem]SYSTEM SHUTDOWN ABORT.
2024-02-21 22:52:52.806 [FATAL] dmasmsvr P0000034610 T0000000000000034613 asvr2_heart_beat_dsk_thread halt. Last timestamp:35520641, sys_status:INVALID EP STATUS, vtd_status:SYSHALT
2024-02-21 22:52:54.808 [FATAL] dmasmsvr P0000034610 T0000000000000034613 [for dem]SYSTEM SHUTDOWN ABORT.
2024-02-21 22:52:54.808 [FATAL] dmasmsvr P0000034610 T0000000000000034613 VTD_CMD_SYS_HALT
2024-02-21 23:01:38.341 [INFO] dmasmsvr P0000034948 T0000000000000034948 DMASMSVR V8
启动成功：
CSS：
2024-02-21 23:17:33:464] [ASM]: CSS set cmd SYS HALT, dest_ep ASM0 seqno = 0, cmd_seq = 60
[2024-02-21 23:17:33:465] [ASM]: Set EP ASM0[0] as Control node
[2024-02-21 23:17:33:466] [ASM]: CSS set cmd START NOTIFY, dest_ep ASM0 seqno = 0, cmd_seq = 62
[2024-02-21 23:18:12:648] [ASM]: CSS set cmd EP START, dest_ep ASM0 seqno = 0, cmd_seq = 63
[2024-02-21 23:18:12:852] [ASM]: CSS set cmd NONE, dest_ep ASM0 seqno = 0, cmd_seq = 0
[2024-02-21 23:18:12:954] [ASM]: CSS set cmd EP START, dest_ep ASM1 seqno = 1, cmd_seq = 65
[2024-02-21 23:18:13:465] [ASM]: CSS set cmd NONE, dest_ep ASM1 seqno = 1, cmd_seq = 0
[2024-02-21 23:18:13:567] [ASM]: CSS set cmd EP OPEN, dest_ep ASM0 seqno = 0, cmd_seq = 72
[2024-02-21 23:18:13:567] [ASM]: CSS set cmd EP OPEN, dest_ep ASM1 seqno = 1, cmd_seq = 73
[2024-02-21 23:18:13:873] [ASM]: CSS set cmd NONE, dest_ep ASM0 seqno = 0, cmd_seq = 0
[2024-02-21 23:18:14:483] [ASM]: CSS set cmd NONE, dest_ep ASM1 seqno = 1, cmd_seq = 0
[2024-02-21 23:18:14:484] [ASM]: CSS set cmd EP REAL OPEN, dest_ep ASM0 seqno = 0, cmd_seq = 75
[2024-02-21 23:18:14:485] [ASM]: CSS set cmd EP REAL OPEN, dest_ep ASM1 seqno = 1, cmd_seq = 76
[2024-02-21 23:18:14:892] [ASM]: CSS set cmd NONE, dest_ep ASM0 seqno = 0, cmd_seq = 0
[2024-02-21 23:18:15:403] [ASM]: CSS set cmd NONE, dest_ep ASM1 seqno = 1, cmd_seq = 0
ASM:
[dmdba@localhost bin]$ /dm8/bin/dmasmsvr DCR_INI=/dm8/dmdbms/dsc_config/dmdcr.ini
ASM SELF EPNO:0
DMASMSVR V8
dmasmsvr task worker thread startup
the ASM server is Ready.
check css cmd: START NOTIFY, cmd_seq: 62
check css cmd: EP START, cmd_seq: 63
ASM Control Node EPNO:0
[WARNING]Decode asmdisk device fail, sig:1751483255, disk_id:65535, group_id:65535.
[WARNING]Decode asmdisk device fail, sig:1751483255, disk_id:65535, group_id:65535.
check css cmd: EP OPEN, cmd_seq: 72
check css cmd: EP REAL OPEN, cmd_seq: 75
说明：如果 DMCSS 配置有自动拉起 dmasmsvr 的功能，可以等待 DMCSS 自动拉起 dmasmsvr 程序，不需要手动启动。
3.13 dmasmtool工具创建redo日志磁盘组和数据文件磁盘组(创建 DMASM 磁盘组)
##在DSC01 节点启动 dmasmtool 工具(一个节点启动即可)
[dmdba@]#/dm8/bin/dmasmtool DCR_INI=/dm8/dmdbms/dsc_config/dmdcr.ini
[dmdba@localhost bin]$ /dm8/bin/dmasmtool DCR_INI=/dm8/dmdbms/dsc_config/dmdcr.ini
DMASMTOOL V8
##创建日志磁盘组
ASM>create diskgroup 'DMLOG' asmdisk '/dev/raw/raw3'
Used time: 77.634(ms).
##创建数据磁盘组
ASM>create diskgroup 'DMDATA' asmdisk '/dev/raw/raw4'
Used time: 58.783(ms).
ASM>
查看磁盘信息：
ASM>ls
+
disk groups total [4]......
NO.1 name: DMLOG
NO.2 name: DMDATA
NO.3 name: VOTE
NO.4 name: DCR
Used time: 1.732(ms).
ASM>
3.14 初始化数据库实例
DMDSC01 和 DMDSC02机器配置相同。
在DSC01节点上执行：
##创建 dminit.ini 文件
root@localhost ~]# vim /dm8/dmdbms/dsc_config/dminit.ini
DB_NAME = DSC
SYSDBA_PWD = SYSDBA_PWD
SYSTEM_PATH = +DMDATA/data
SYSTEM = +DMDATA/data/SYSTEM.dbf
SYSTEM_SIZE = 128
ROLL = +DMDATA/data/ROLL.dbf
ROLL_SIZE = 10000
MAIN = +DMDATA/data/MAIN.dbf
MAIN_SIZE = 1024
CTL_PATH = +DMDATA/data/dm.ctl
CTL_SIZE = 8
LOG_SIZE = 2048
DCR_PATH = /dev/raw/raw1 #dcr 磁盘路径
DCR_SEQNO = 0
AUTO_OVERWRITE = 1
PAGE_SIZE = 32
EXTENT_SIZE = 16
BLANK_PAD_MODE = 1
[DSC0] #inst_name 跟 dmdcr_cfg.ini 中 DB 类型 group 中 DCR_EP_NAME 对应
CONFIG_PATH = /dm8/dmdbms/dsc_config/DSC0
PORT_NUM = 5236
MAL_HOST = 192.168.145.128
MAL_PORT = 11246
LOG_PATH = +DMLOG/DSC0_LOG01.log
LOG_PATH = +DMLOG/DSC0_LOG02.log
[DSC1]
CONFIG_PATH = /dm8/dmdbms/dsc_config/DSC1
PORT_NUM = 5237
MAL_HOST = 192.168.145.129
MAL_PORT = 11247
LOG_PATH = +DMLOG/DSC1_LOG01.log
LOG_PATH = +DMLOG/DSC1_LOG02.log
初始化实例失败：
/dm8/bin/dminit control=/dm8/dmdbms/dsc_config/dminit.ini
ormal of FAST
Normal of DEFAULT
Normal of RECYCLE
Normal of KEEP
Normal of ROLL
log file path: +DMLOG/DSC0_LOG01.log
log file path: +DMLOG/DSC0_LOG02.log
log file path: +DMLOG/DSC1_LOG01.log
log file path: +DMLOG/DSC1_LOG02.log
write to dir [+DMDATA/data/DSC].
create rlog file +DMLOG/DSC0_LOG01.log failed, code: -7013.
You may get more details in file ../log/dm_DSC0_202402.log
fail to init db.
查看日志，out of space:
[dmdba@localhost log]$ cat dm_DSC0_202402.log
2024-02-21 23:45:45.131 [INFO] dminit P0000036035 T0000000000000036035 os_sema2_create_low, create and inc sema success, key:119977218, sem_id:4, sem_value:1!
2024-02-21 23:45:45.132 [INFO] dminit P0000036035 T0000000000000036035 os_sema2_create_low, create and inc sema success, key:388412674, sem_id:5, sem_value:1!
2024-02-21 23:45:45.132 [ERROR] dminit P0000036035 T0000000000000036035 os_dir_create_low failed! path: /dm, code: 13, desc: Permission denied
2024-02-21 23:45:45.132 [INFO] dminit P0000036035 T0000000000000036035 dmshm2_detach, ret = 0 shm id 98330
2024-02-21 23:45:45.133 [FATAL] dminit P0000036035 T0000000000000036035 init database fail with code -1!
2024-02-21 23:49:43.489 [INFO] dminit P0000036163 T0000000000000036163 os_sema2_create_low, create and inc sema success, key:119977218, sem_id:4, sem_value:1!
2024-02-21 23:49:43.489 [INFO] dminit P0000036163 T0000000000000036163 os_sema2_create_low, create and inc sema success, key:388412674, sem_id:5, sem_value:1!
2024-02-21 23:49:43.491 [INFO] dminit P0000036163 T0000000000000036163 INI parameter IO_THR_GROUPS changed, the original value 0, new value 8
2024-02-21 23:49:43.497 [INFO] dminit P0000036163 T0000000000000036163 INI parameter FAST_POOL_PAGES changed, the original value 0, new value 16
2024-02-21 23:49:43.497 [INFO] dminit P0000036163 T0000000000000036163 INI parameter BUFFER_POOLS changed, the original value 0, new value 1
2024-02-21 23:49:43.497 [INFO] dminit P0000036163 T0000000000000036163 INI parameter RECYCLE_POOLS changed, the original value 0, new value 1
2024-02-21 23:49:43.497 [INFO] dminit P0000036163 T0000000000000036163 INI parameter ROLLSEG_POOLS changed, the original value 0, new value 1
2024-02-21 23:49:43.499 [INFO] dminit P0000036163 T0000000000000036163 fil_sys_init
2024-02-21 23:49:43.499 [INFO] dminit P0000036163 T0000000000000036163 init database start at +DMDATA/data/DSC
2024-02-21 23:49:45.027 [INFO] dminit P0000036163 T0000000000000036163 ctl_write_to_file file: +DMDATA/data/dm.ctl
2024-02-21 23:49:45.802 [INFO] dminit P0000036163 T0000000000000036163 ctl_add_table_space_low to lst success, ts_name RLOG[id=2]
2024-02-21 23:49:45.807 [ERROR] dminit P0000036163 T0000000000000036163 os_file_create_with_init->os_asm_file_create: [path: +DMLOG/DSC0_LOG01.log]: [CODE:-523] Out of space
2024-02-21 23:49:45.807 [INFO] dminit P0000036163 T0000000000000036163 os_sema2_free, sema_id:4, sema_value:1!
2024-02-21 23:49:45.807 [INFO] dminit P0000036163 T0000000000000036163 os_sema2_free, sema_id:5, sema_value:1!
2024-02-21 23:49:45.808 [INFO] dminit P0000036163 T0000000000000036163 dmshm2_detach, ret = 0 shm id 98330
2024-02-21 23:49:45.810 [FATAL] dminit P0000036163 T0000000000000036163 init database fail with code -2!
修改初始化日志的大小，调整dminit.ini参数
root@localhost ~]# vim /dm8/dmdbms/dsc_config/dminit.ini
root@localhost ~]# vim /dm8/dmdbms/dsc_config/dminit.ini
DB_NAME = DSC
SYSDBA_PWD = SYSDBA_PWD
SYSTEM_PATH = +DMDATA/data
SYSTEM = +DMDATA/data/SYSTEM.dbf
SYSTEM_SIZE = 128
ROLL = +DMDATA/data/ROLL.dbf
ROLL_SIZE = 128
MAIN = +DMDATA/data/MAIN.dbf
MAIN_SIZE = 128
CTL_PATH = +DMDATA/data/dm.ctl
CTL_SIZE = 8
LOG_SIZE = 256
DCR_PATH = /dev/raw/raw1 #dcr 磁盘路径
DCR_SEQNO = 0
AUTO_OVERWRITE = 1
PAGE_SIZE = 32
EXTENT_SIZE = 16
BLANK_PAD_MODE = 1
[DSC0] #inst_name 跟 dmdcr_cfg.ini 中 DB 类型 group 中 DCR_EP_NAME 对应
CONFIG_PATH = /dm8/dmdbms/dsc_config/DSC0
PORT_NUM = 5236
MAL_HOST = 192.168.145.128
MAL_PORT = 11246
LOG_PATH = +DMLOG/DSC0_LOG01.log
LOG_PATH = +DMLOG/DSC0_LOG02.log
[DSC1]
CONFIG_PATH = /dm8/dmdbms/dsc_config/DSC1
PORT_NUM = 5237
MAL_HOST = 192.168.145.129
MAL_PORT = 11247
LOG_PATH = +DMLOG/DSC1_LOG01.log
LOG_PATH = +DMLOG/DSC1_LOG02.log
初始化 db 命令，初始化实例成功：
[dmdba@localhost ~]$ /dm8/bin/dminit control=/dm8/dmdbms/dsc_config/dminit.ini
initdb V8
db version: 0x7000c
file dm.key not found, use default license!
License will expire on 2024-12-26
Normal of FAST
Normal of DEFAULT
Normal of RECYCLE
Normal of KEEP
Normal of ROLL
log file path: +DMLOG/DSC0_LOG01.log
log file path: +DMLOG/DSC0_LOG02.log
log file path: +DMLOG/DSC1_LOG01.log
log file path: +DMLOG/DSC1_LOG02.log
FILE "/dm8/dmdbms/dsc_config/DSC0/dm.ini" has already existed
FILE "/dm8/dmdbms/dsc_config/DSC0/sqllog.ini" has already existed
FILE "/dm8/dmdbms/dsc_config/DSC1/dm.ini" has already existed
FILE "/dm8/dmdbms/dsc_config/DSC1/sqllog.ini" has already existed
FILE "+DMDATA/data/dm.ctl" has already existed
write to dir [+DMDATA/data/DSC].
create dm database success. 2024-02-21 23:58:31
[dmdba@localhost ~]$
拷贝文件到DSC02 节点
mdba@localhost dsc_config]$ scp -r /dm8/dmdbms/dsc_config/DSC1 dmdba@192.168.145.129:/dm8/dmdbms/dsc_config/
The authenticity of host '192.168.145.129 (192.168.145.129)' can't be established.
ECDSA key fingerprint is SHA256:hrYPlFmPCrJTxCmEFykyv7f0qf9okZgKHBE7iHv4g2w.
ECDSA key fingerprint is MD5:cd:7a:9f:c5:3c:7e:3d:9b:fb:ef:fa:96:73:56:25:5a.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.145.129' (ECDSA) to the list of known hosts.
dmdba@192.168.145.129's password:
dm.ini 100% 74KB 42.1MB/s 00:00
sqllog.ini 100% 714 1.7MB/s 00:00
dmmal.ini 100% 212 501.9KB/s 00:00
[dmdba@localhost dsc_config]$
DSC02节点查看
ot@localhost log]# cd /dm8/dmdbms/dsc_config
[root@localhost dsc_config]# ll
total 12
-rw-rw-r--. 1 dmdba dmdba 267 Feb 21 14:01 dmasvrmal.ini
-rw-rw-r--. 1 dmdba dmdba 2057 Feb 20 13:27 dmdcr_cfg.ini
-rw-rw-r--. 1 dmdba dmdba 861 Feb 21 14:45 dmdcr.ini
drwxrwxr-x. 2 dmdba dmdba 55 Feb 21 16:04 DSC1
[root@localhost dsc_config]#
3.15 启动css，asm，server
两个节点都需要
[dmdba@localhost bin]$ ./dmcss dcr_ini=/dm8/dmdbms/dsc_config/dmdcr.ini --css前面没关它，就不需要重启
[dmdba@localhost bin]$ /dm8/bin/dmasmsvr DCR_INI=/dm8/dmdbms/dsc_config/dmdcr.ini --asm这里后台已经全部
启动DSC01节点失败：
[dmdba@localhost bin]$ /dm8/bin/dmserver path=/dm8/dmdbms/dsc_config/DSC0/dm.ini dcr_ini=/dm8/dmdbms/dsc_config/dmdcr.ini
file dm.key not found, use default license!
version info: develop
DM Database Server 64 V8 03134284132-20231226-213242-20081 startup...
Can not create buffer pools, out of memory!
[dmdba@localhost bin]$
减小dm.ini里面的buffer相关值（如果两个虚拟机配置一致，两个虚拟机都需要减少）
再次启动实例，注意要两个实例都启动，不要启动一个，然后一直等着它启动成功，你是等不到的。
两个实例启动成功的标志，system is ready.
3.16 注册 CSS、ASM后台服务
两个节点均注册：
[root@localhost dm8]# /dm8/script/root/dm_service_installer.sh -t dmcss -dcr_ini /dm8/dmdbms/dsc_config/dmdcr.ini -p CSS
Created symlink from /etc/systemd/system/multi-user.target.wants/DmCSSServiceCSS.service to /usr/lib/systemd/system/DmCSSServiceCSS.service.
创建服务(DmCSSServiceCSS)完成
[root@localhost dm8]# dm8/script/root/dm_service_installer.sh -t dmasmsvr -dcr_ini /dm8/dmdbms/dsc_config/dmdcr.ini -y DmCSSServiceCSS.service -p ASM
-bash: dm8/script/root/dm_service_installer.sh: No such file or directory
[root@localhost dm8]# /dm8/script/root/dm_service_installer.sh -t dmasmsvr -dcr_ini /dm8/dmdbms/dsc_config/dmdcr.ini -y DmCSSServiceCSS.service -p ASM
Created symlink from /etc/systemd/system/multi-user.target.wants/DmASMSvrServiceASM.service to /usr/lib/systemd/system/DmASMSvrServiceASM.service.
创建服务(DmASMSvrServiceASM)完成
3.17 配置监视器：
两个节点均配置：内容一致
[dmdba@localhost ~]$ vim /dm8/bin/dmcssm.ini
#和 dmdcr_cfg.ini 中的 DCR_OGUID 保持一致
CSSM_OGUID = 63635
#配置所有 CSS 的连接信息，
#和 dmdcr_cfg.ini 中 CSS 配置项的 DCR_EP_HOST 和 DCR_EP_PORT 保持一致
CSSM_OGUID = 210715
CSSM_CSS_IP = 192.168.145.128:11286
CSSM_CSS_IP = 1192.168.145.129:11287
CSSM_LOG_PATH = ../log
CSSM_LOG_FILE_SIZE = 128
CSSM_LOG_SPACE_LIMIT = 256
注册服务：
[root@localhost ~]# /dm8/script/root/dm_service_installer.sh -t dmcssm -cssm_ini /dm8/bin/dmcssm.ini -p Monitor
Created symlink from /etc/systemd/system/multi-user.target.wants/DmCSSMonitorServiceMonitor.service to /usr/lib/systemd/system/DmCSSMonitorServiceMonitor.service.
创建服务(DmCSSMonitorServiceMonitor)完成
[root@localhost ~]#
3.18 启动监视器
3.19 客户端验证：
在DSC01上：
[dmdba@localhost bin]$ ./disql SYSDBA/SYSDBA@192.168.145.128:
[-70064]:Invalid config value.
disql V8
username:SYSDBA
password:
Server[LOCALHOST:5236]:mode is normal, state is open
login used time : 27.840(ms)
SQL> select * from v$dsc_ep_info;
LINEID EP_NAME EP_SEQNO EP_GUID EP_TIMESTAMP EP_MODE EP_STATUS
---------- ------- ----------- -------------------- -------------------- ------------ ---------
1 DSC0 0 57520699 57525615 Control Node OK
2 DSC1 1 109760264 109764994 Normal Node OK
used time: 13.206(ms). Execute id is 62601.
SQL> create table table_3 (a char(10));
executed successfully
used time: 55.536(ms). Execute id is 62602.
在DSC02节点上：
[dmdba@localhost bin]$ ./disql SYSDBA/SYSDBA_PWD@192.168.145.129:5237
Server[192.168.145.129:5237]:mode is normal, state is open
login used time : 5.257(ms)
disql V8
SQL> select * from v$dsc_ep_info;
LINEID EP_NAME EP_SEQNO EP_GUID EP_TIMESTAMP
---------- ------- ----------- -------------------- --------------------
EP_MODE EP_STATUS
------------ ---------
1 DSC0 0 57520699 57525952
Control Node OK
2 DSC1 1 109760264 109765331
Normal Node OK
used time: 5.872(ms). Execute id is 201.
SQL> select * from table table_3;
select * from table table_3;
select * from table table_3;
*
line 1, column 42, nearby [table_3] has error[-2007]:
Syntax error.
used time: 1.707(ms). Execute id is 0.
SQL> select * from table_3;
no rows
used time: 6.940(ms). Execute id is 202.
SQL>
