【Kingbase FlySync】命令模式:安装部署同步软件，实现Oracle到KES实现同步
- 双轨并行方案说明
- 一.准备工作
- 二.环境说明
- 三.目标
- 实操
- (1).准备安装环境
- Orcle服务器(Oracle40)
- 1.上传所有工具包
- 2.操作系统配置
- a.增加flysync 用户、设置密码
- b.配置环境变量
- c.调整limits.conf
- d.hosts文件
- e.配置hostname
- f.授权flysync访问redo
- g.安装ruby工具
- 3.数据库配置
- a.启动ORACLE监听
- b.启动ORACLE服务，并将服务注册到监听程序
- c.创建flysync 用户并授权
- d.验证flysync 用户权限
- e.oracle 密码过期处理
- f.开启归档、开启补全日志 (建议在 MOUNT 模式下执行)。
- g.再次查看数据库归档情况、检查补全日志的状态。
- 4.环境检查
- KingbaseES服务器(node2)
- 1.上传所有工具包
- 2.操作系统配置
- a.增加flysync 用户、设置密码
- b.调整limits.conf
- c.hosts文件
- d.配置hostname
- e.安装ruby工具
- 3.数据库配置
- a.创建flysync用户
- b.配置数据参数
- c.修改sys_hba.conf文件
- d.重启数据库
- 4.环境检查
- (2).部署同步程序
- Oracle端（Oracle40）部署抽取程序
- 1.解压安装包并拷贝license
- 2.静默安装
- a.配置数据库驱动包
- b.创建Kingbase FlySync安装配置文件(flysync.ini)
- c.配置flysync.ini文件
- d.执行安装
- f.更换license
- g.生效环境变量
- h.配置DML映射策略(作为目标端)
- i.启动同步程序并置于offline 状态
- j.查看同步软件的运行状态
- k.配置双轨切换配置文件
- KES端(node2)部署同步程序
- 1.解压安装包并拷贝license
- 2.静默安装
- 1.解压安装包并拷贝license
- 2.静默安装
- a.配置数据库驱动包
- b.创建Kingbase FlySync安装配置文件(flysync.ini)
- c.配置flysync.ini文件
- d.执行安装
- f.更换license
- g.生效环境变量
- h.配置DML映射策略(作为目标端)
- i.启动同步程序并置于offline 状态
- j.查看同步软件的运行状态
- k.配置双轨切换配置文件
- 指定同步角色（Oracle 为主，kingbase为备)
- 1.ONLINE同步服务，数据开始同步
- 2.检查同步进程状态
- a.Oracle端(oracle40)
- b.KES端(node2)
- (3).KFS存量数据搬迁
- 1.检查replicator状态
- a.Oracle端(oracle40)
- b.KES端(node2)
- 2.在node2上创建oa11g0017模式
- 3.用ddlscan极速模式/平滑模式（目标端流水线）迁移结构数据。
- 4.用loader极速模式进行数据搬迁
- (4).增量数据同步(Oracle为主，Kingbase 为备)
- 1.指定主库和备库，ONLINE同步服务，数据开始同步
- 2.查看同步进程状态
- a.Oracle端(oracle40)
- b.KES端(node2)
- 3.数据同步验证(验证DML)
- a.在源端（oracle40）生成数据
- b.查看源端（oracle40）kufl 是否解析
- c.在目标端（node2）验证kufl 已装载。
- d.在目标端（node2）验证数据已同步
- (5).增量数据同步(kingbase为主，Oracle 为备)
- 1.指定主库和备库，ONLINE同步服务，数据开始同步
- 2.查看同步进程状态
- a.Oracle端(oracle40)
- b.KES端(node2)
- 3.数据同步验证(验证DML)
- a.在源端（node2）生成数据
- b.查看源端（node2）kufl 是否解析
- c.在目标端（oracle40）验证kufl 已装载。
- d.在目标端（oracle40）验证数据已同步
双轨并行方案说明
- KFS双轨指的是KFS的源端和目标端可以自由切换。通过命令切换后，源端变为目标端，目标端变为源端，不需要修改任何配置。
- KFS双轨只适合一对一的场景，并且要求必须安装两个replicator,且两端服务的角色必须为master，使用repswitch命令来确定主从关系。
- KFS双轨一般是用来进行国产化替代场景里，当国产数据库替代国外数据库不顺利，可以很方便地将业务转回国外数据库，保持数据不丢失。并且在国产数据库解决问题后，再方便切换回去，数据不用重新迁移。
一.准备工作
- Oracle40虚拟机还原至【优化数据库】场景。
- node2虚拟机还原至【优化数据库】场景。
二.环境说明
- 两台虚拟机已安装好数据库
| 虚拟机 | IP | 数据库软件 | 角色 | 操作系统版本 |
|---|---|---|---|---|
| Oracle40 | 192.168.40.40 | Oracle11g | 主库 | CenterOS7.2 |
| node2 | 192.168.40.112 | KingbaseES V8R6 | 备库 | CenterOS7.2 |
- 系统环境详细信息
| 类别 | 源端 | 目标端 |
|---|---|---|
| IP | 192.168.40.40 | 192.168.40.112 |
| 主机名 | Oracle40 | node2 |
| os用户1/密码 | root/kingbase | root/kingbase.123 |
| os用户2/密码 | oracle/oracle | kingbase/kingbase |
- 数据库环境详细信息
| 类别 | 源端 | 目标端 |
|---|---|---|
| 数据库软件 | Oracle11g | KingbaseES V8R6 |
| 数据库名 | orcl | test |
| 数据库端口号 | 1521 | 54321 |
| 数据库用户 | sys/oracle | system/kingbase |
三.目标
| 类别 | 源端 | 目标端 |
|---|---|---|
| replicater安装路径 | /home/flysync/replicator | /home/flysync/replicator |
| replicator rmi-port端口 | 11000 | 11000 |
| replicator kufl-port端口 | 3112 | 3112 |
| kfs服务名 | oracle_source | kingbase8_target |
| KFS 同步所使用的用户/密码 | FLYSYNC/flysync | flysync/flysync |
实操
(1).准备安装环境
Orcle服务器(Oracle40)
1.上传所有工具包
[root@oracle40 ~]# mkdir /install
[root@oracle40 install]# ls -l
total 146552
-rw-r--r-- 1 root root 143069019 Nov 17 21:39 KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
-rw-r--r-- 1 root root 1396 Nov 17 21:39 license.dat
-rw-r--r-- 1 root root 6993781 Nov 17 21:38 rbenv_2.2.2_linux_x64.zip
[root@oracle40 install]# chown -R flysync:flysync /install
[root@oracle40 install]#
2.操作系统配置
a.增加flysync 用户、设置密码
[root@oracle40 ~]# adduser flysync
[root@oracle40 ~]# usermod -G oinstall,dba flysync
[root@oracle40 ~]# echo flysync | passwd --stdin flysync
Changing password for user flysync.
passwd: all authentication tokens updated successfully.
[root@oracle40 ~]#
b.配置环境变量
增加内容：
export ORACLE_SID=orcl
export ORACLE_HOME=/home/oracle/app/oracle/product/11.2.0/dbhome_1
[root@oracle40 ~]# vi /home/flysync/.bash_profile
[root@oracle40 ~]# cat /home/flysync/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export ORACLE_SID=orcl
export ORACLE_HOME=/home/oracle/app/oracle/product/11.2.0/dbhome_1
export PATH
[root@oracle40 ~]# source /home/flysync/.bash_profile
[root@oracle40 ~]#
c.调整limits.conf
limits.conf
最大文件句柄数（open files）、允许创建的最大进程数量（max user processes）
新增内容：
flysync - nofile 65535
flysync - nproc 8096
[root@oracle40 ~]# vi /etc/security/limits.conf
[root@oracle40 ~]# grep flysync /etc/security/limits.conf
flysync - nofile 65535
flysync - nproc 8096
[root@oracle40 ~]#
d.hosts文件
新增内容
192.168.40.112 node2
[root@oracle40 ~]# vi /etc/hosts
[root@oracle40 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.40.40 oracle40
192.168.40.112 node2
[root@oracle40 ~]#
e.配置hostname
/etc/sysconfig/network内容改为
NETWORKING = yes
HOSTNAME = oracle40
[root@oracle40 ~]# vi /etc/sysconfig/network
[root@oracle40 ~]# cat /etc/sysconfig/network
NETWORKING = yes
HOSTNAME = oracle40
[root@oracle40 ~]#
f.授权flysync访问redo
[root@oracle40 ~]# chmod g+r,g+x /home/oracle
[root@oracle40 ~]#
g.安装ruby工具
[root@oracle40 ~]# su - flysync
Last login: Fri Nov 17 21:45:45 CST 2023 on pts/0
[flysync@oracle40 ~]$ unzip /install/rbenv_2.2.2_linux_x64.zip -d ~/
[flysync@oracle40 ~]$ ls -l
total 4
-rwxr-xr-x 1 flysync flysync 340 Nov 4 2019 fspm
drwxrwxr-x 3 flysync flysync 17 Nov 4 2019 rbenv
[flysync@oracle40 ~]$ vi ~/.bash_profile
[flysync@oracle40 ~]$ cat ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export ORACLE_SID=orcl
export ORACLE_HOME=/home/oracle/app/oracle/product/11.2.0/dbhome_1
export RUBY_HOME=/home/flysync/rbenv/ruby
export PATH=$RUBY_HOME/bin:$PATH
export PATH
[flysync@oracle40 ~]$ source ~/.bash_profile
[flysync@oracle40 ~]$ ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
[flysync@oracle40 ~]$
~/.bash_profile文件增加内容：
export RUBY_HOME=/home/flysync/rbenv/ruby
export PATH=$RUBY_HOME/bin:$PATH
3.数据库配置
a.启动ORACLE监听
[flysync@oracle40 ~]$ exit
logout
[root@oracle40 ~]# su - oracle
Last login: Thu Apr 13 09:53:52 CST 2023 on pts/0
[oracle@oracle40 ~]$ lsnrctl start
LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 17-NOV-2023 21:57:10
Copyright (c) 1991, 2013, Oracle. All rights reserved.
Starting /home/oracle/app/oracle/product/11.2.0/dbhome_1/bin/tnslsnr: please wait...
TNSLSNR for Linux: Version 11.2.0.4.0 - Production
Log messages written to /home/oracle/app/oracle/product/11.2.0/dbhome_1/log/diag/tnslsnr/oracle40/listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oracle40)(PORT=1521)))
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.4.0 - Production
Start Date 17-NOV-2023 21:57:10
Uptime 0 days 0 hr. 0 min. 0 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Log File /home/oracle/app/oracle/product/11.2.0/dbhome_1/log/diag/tnslsnr/oracle40/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oracle40)(PORT=1521)))
The listener supports no services
The command completed successfully
[oracle@oracle40 ~]$
b.启动ORACLE服务，并将服务注册到监听程序
[oracle@oracle40 ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.4.0 Production on Fri Nov 17 22:00:48 2023
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to an idle instance.
SQL> startup open
ORACLE instance started.
Total System Global Area 789172224 bytes
Fixed Size 2257392 bytes
Variable Size 515902992 bytes
Database Buffers 264241152 bytes
Redo Buffers 6770688 bytes
Database mounted.
Database opened.
SQL> alter system register;
System altered.
SQL>
c.创建flysync 用户并授权
SQL> CREATE USER FLYSYNC IDENTIFIED BY flysync;
User created.
SQL> GRANT CONNECT TO flysync;
Grant succeeded.
SQL> GRANT CREATE SESSION TO flysync;
Grant succeeded.
SQL> GRANT UNLIMITED TABLESPACE TO flysync;
Grant succeeded.
SQL> GRANT CREATE TABLE TO flysync;
Grant succeeded.
SQL> GRANT EXECUTE_CATALOG_ROLE TO flysync;
Grant succeeded.
SQL> GRANT SELECT ANY DICTIONARY TO flysync;
Grant succeeded.
SQL> GRANT SELECT ON V_$PARAMETER TO flysync;
Grant succeeded.
SQL> GRANT SELECT ANY TABLE TO flysync;
Grant succeeded.
SQL> GRANT SELECT ANY TRANSACTION TO flysync;
Grant succeeded.
SQL>
d.验证flysync 用户权限
SQL> exit
[oracle@oracle40 ~]$ cp /install/privs.sql ~/
[oracle@oracle40 ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.4.0 Production on Fri Nov 17 22:14:22 2023
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> @privs.sql FLYSYNC
TYPE GRANTEE PRIVILEGE AD TABLE_NAME COLUMN_NAME OWNER
------ --------------- ------------------------------ --- ------------------------------ --------------- --------------------
ROLE FLYSYNC CONNECT NO / / /
ROLE FLYSYNC EXECUTE_CATALOG_ROLE NO / / /
SYSTEM FLYSYNC CREATE SESSION NO / / /
SYSTEM FLYSYNC CREATE TABLE NO / / /
SYSTEM FLYSYNC SELECT ANY DICTIONARY NO / / /
SYSTEM FLYSYNC SELECT ANY TABLE NO / / /
SYSTEM FLYSYNC SELECT ANY TRANSACTION NO / / /
SYSTEM FLYSYNC UNLIMITED TABLESPACE NO / / /
TABLE FLYSYNC SELECT NO V_$PARAMETER / SYS
9 rows selected.
SQL>
e.oracle 密码过期处理
SQL> ALTER USER OA11G identified by oa11g ACCOUNT UNLOCK;
User altered.
f.开启归档、开启补全日志 (建议在 MOUNT 模式下执行)。
SQL> SHUTDOWN immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> STARTUP MOUNT;
ORACLE instance started.
Total System Global Area 789172224 bytes
Fixed Size 2257392 bytes
Variable Size 515902992 bytes
Database Buffers 264241152 bytes
Redo Buffers 6770688 bytes
Database mounted.
SQL> ALTER DATABASE archivelog;
Database altered.
SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
Database altered.
SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS;
Database altered.
SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;
Database altered.
SQL> ALTER DATABASE OPEN;
Database altered.
SQL>
g.再次查看数据库归档情况、检查补全日志的状态。
SQL> SELECT log_mode FROM v$database;
LOG_MODE
------------
ARCHIVELOG
SQL> SELECT supplemental_log_data_min, supplemental_log_data_pk,
supplemental_log_data_all FROM v$database; 2
SUPPLEME SUP SUP
-------- --- ---
YES YES YES
SQL>
4.环境检查
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
[oracle@oracle40 ~]$ exit
logout
[root@oracle40 ~]# su - flysync
Last login: Fri Nov 17 21:49:13 CST 2023 on pts/0
[flysync@oracle40 ~]$ uname -n
oracle40
[flysync@oracle40 ~]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 11210
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 8096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[flysync@oracle40 ~]$ hostname --ip-address
192.168.40.40
[flysync@oracle40 ~]$ ping -c 4 oracle40
PING oracle40 (192.168.40.40) 56(84) bytes of data.
64 bytes from oracle40 (192.168.40.40): icmp_seq=1 ttl=64 time=0.035 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq=2 ttl=64 time=0.046 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq=3 ttl=64 time=0.037 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq=4 ttl=64 time=0.044 ms
--- oracle40 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.035/0.040/0.046/0.007 ms
[flysync@oracle40 ~]$ ping -c 4 node2
PING node2 (192.168.40.112) 56(84) bytes of data.
64 bytes from node2 (192.168.40.112): icmp_seq=1 ttl=64 time=1.60 ms
64 bytes from node2 (192.168.40.112): icmp_seq=2 ttl=64 time=0.320 ms
64 bytes from node2 (192.168.40.112): icmp_seq=3 ttl=64 time=0.253 ms
64 bytes from node2 (192.168.40.112): icmp_seq=4 ttl=64 time=0.227 ms
--- node2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3001ms
rtt min/avg/max/mdev = 0.227/0.602/1.609/0.582 ms
[root@oracle40 ~]# date
Fri Nov 17 22:24:39 CST 2023
[root@oracle40 ~]# ssh root@node2 date
The authenticity of host 'node2 (192.168.40.112)' can't be established.
ECDSA key fingerprint is 03:d4:89:aa:aa:7c:e5:b4:c2:ca:09:8f:c7:63:19:a3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node2,192.168.40.112' (ECDSA) to the list of known hosts.
root@node2's password:
Fri Nov 17 22:25:00 CST 2023
[root@oracle40 ~]#
KingbaseES服务器(node2)
1.上传所有工具包
[root@node2 install]# ls -l
总用量 998884
-rw-r--r-- 1 root root 872781824 4月 9 2023 KingbaseES_V008R006C005B0023_Lin64_single_install.iso
-rw-r--r-- 1 root root 143069019 11月 17 22:34 KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
-rw-r--r-- 1 root root 3351 4月 9 2023 license_12350_0_dev.dat
-rw-r--r-- 1 root root 1396 11月 17 22:34 license.dat
-rw-r--r-- 1 root root 6993781 11月 17 22:34 rbenv_2.2.2_linux_x64.zip
drwxrwxr-x 2 kingbase kingbase 6 4月 9 2023 scripts
[root@node2 install]# chown -R flysync:flysync /install
[root@node2 install]#
2.操作系统配置
a.增加flysync 用户、设置密码
[root@node2 install]# adduser flysync
[root@node2 install]# usermod -G kingbase flysync
[root@node2 install]# echo flysync |passwd --stdin flysync
更改用户 flysync 的密码 。
passwd：所有的身份验证令牌已经成功更新。
[root@node2 install]#
b.调整limits.conf
limits.conf
最大文件句柄数（open files）、允许创建的最大进程数量（max user processes）
新增内容：
flysync - nofile 65535
flysync - nproc 8096
[root@node2 install]# vi /etc/security/limits.conf
[root@node2 install]# grep flysync /etc/security/limits.conf
flysync - nofile 65535
flysync - nproc 8096
[root@node2 install]#
c.hosts文件
新增内容
192.168.40.40 oracle40
192.168.40.112 node2
[root@node2 install]# vi /etc/hosts
[root@node2 install]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.40.40 oracle40
192.168.40.112 node2
[root@node2 install]#
d.配置hostname
/etc/sysconfig/network内容改为
NETWORKING = yes
HOSTNAME = node2
[root@node2 install]# vi /etc/sysconfig/network
[root@node2 install]# cat /etc/sysconfig/network
NETWORKING = yes
HOSTNAME = node2
[root@node2 install]#
e.安装ruby工具
[root@node2 ~]# su - flysync
Last login: Fri Nov 17 21:45:45 CST 2023 on pts/0
[flysync@node2 ~]$ unzip /install/rbenv_2.2.2_linux_x64.zip -d ~/
[flysync@node2 ~]$ ls -l
总用量 4
-rwxr-xr-x 1 flysync flysync 340 11月 4 2019 fspm
drwxrwxr-x 3 flysync flysync 17 11月 4 2019 rbenv
[flysync@node2 ~]$
[flysync@node2 ~]$ vi ~/.bash_profile
[flysync@node2 ~]$ cat ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export RUBY_HOME=/home/flysync/rbenv/ruby
export PATH=$RUBY_HOME/bin:$PATH
export PATH
[flysync@node2 ~]$ source ~/.bash_profile
[flysync@node2 ~]$ ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
[flysync@node2 ~]$
~/.bash_profile文件增加内容：
export RUBY_HOME=/home/flysync/rbenv/ruby
export PATH=$RUBY_HOME/bin:$PATH
3.数据库配置
a.创建flysync用户
[flysync@node2 ~]$ exit
登出
[root@node2 install]# su - kingbase
上一次登录：五 11月 17 21:19:58 CST 2023
[kingbase@node2 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbase@node2 bin]$ ./ksql -Usystem test
ksql (V8.0)
输入 "help" 来获取帮助信息.
test=# CREATE USER flysync PASSWORD 'flysync';
CREATE ROLE
test=# GRANT ALL PRIVILEGES ON DATABASE test TO flysync ;
GRANT
test=# ALTER USER flysync SUPERUSER;
ALTER ROLE
test=#
b.配置数据参数
test=# ALTER SYSTEM SET wal_level = 'logical';
ALTER SYSTEM
test=# ALTER SYSTEM SET wal_keep_segments = 4;
ALTER SYSTEM
test=# ALTER SYSTEM SET max_wal_senders = 10;
ALTER SYSTEM
test=# ALTER SYSTEM SET max_replication_slots = 10;
ALTER SYSTEM
test=#
c.修改sys_hba.conf文件
test=# \q
[kingbase@node2 ~]$ cat >> /data/sys_hba.conf << EOF
> host replication all 0.0.0.0/0 scram-sha-256
> EOF
d.重启数据库
[kingbase@node2 ~]$ sys_ctl restart -D /data/
waiting for server to shut down.... done
server stopped
waiting for server to start....2023-11-18 15:06:16.999 GMT [39862] 警告: max_connections should be less than or equal than 10 (restricted by license)
2023-11-18 15:06:16.999 GMT [39862] 提示: the value of max_connect is set 10
2023-11-18 15:06:17.000 GMT [39862] 警告: max_connections should be less than or equal than 10 (restricted by license)
2023-11-18 15:06:17.000 GMT [39862] 提示: the value of max_connect is set 10
2023-11-18 23:06:17.007 CST [39862] 日志: sepapower extension initialized
2023-11-18 23:06:17.010 CST [39862] 日志: 正在启动 KingbaseES V008R006C005B0023 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46), 64-bit
2023-11-18 23:06:17.010 CST [39862] 日志: 正在监听IPv4地址"0.0.0.0"，端口 54321
2023-11-18 23:06:17.010 CST [39862] 日志: 正在监听IPv6地址"::"，端口 54321
2023-11-18 23:06:17.014 CST [39862] 日志: 在Unix套接字 "/tmp/.s.KINGBASE.54321"上侦听
2023-11-18 23:06:17.052 CST [39862] 日志: 日志输出重定向到日志收集进程
2023-11-18 23:06:17.052 CST [39862] 提示: 后续的日志输出将出现在目录 "sys_log"中.
done
server started
[kingbase@node2 ~]$
4.环境检查
test=# \q
[kingbase@node2 bin]$ exit
登出
[root@node2 install]# uname -n
node2
[root@node2 install]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 9838
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 9838
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[root@node2 install]# hostname --ip-address
192.168.40.112
[root@node2 install]# ping -c 4 node2
PING node2 (192.168.40.112) 56(84) bytes of data.
64 bytes from node2 (192.168.40.112): icmp_seq=1 ttl=64 time=0.031 ms
64 bytes from node2 (192.168.40.112): icmp_seq=2 ttl=64 time=0.045 ms
64 bytes from node2 (192.168.40.112): icmp_seq=3 ttl=64 time=0.037 ms
64 bytes from node2 (192.168.40.112): icmp_seq=4 ttl=64 time=0.036 ms
--- node2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.031/0.037/0.045/0.006 ms
[root@node2 install]# ping -c 4 oracle40
PING oracle40 (192.168.40.40) 56(84) bytes of data.
64 bytes from oracle40 (192.168.40.40): icmp_seq=1 ttl=64 time=1.24 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq=2 ttl=64 time=0.298 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq=3 ttl=64 time=0.242 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq=4 ttl=64 time=0.281 ms
--- oracle40 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3001ms
rtt min/avg/max/mdev = 0.242/0.517/1.248/0.422 ms
[root@node2 install]# date
2023年 11月 18日 星期六 00:19:27 CST
[root@node2 install]#
(2).部署同步程序
Oracle端（Oracle40）部署抽取程序
1.解压安装包并拷贝license
[root@oracle40 ~]# su - flysync
Last login: Sat Nov 18 22:50:03 CST 2023 on pts/0
[flysync@oracle40 ~]$ mkdir -p /home/flysync/install
[flysync@oracle40 ~]$ exit
logout
[root@oracle40 ~]# cp /install/KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz /home/flysync/install/
[root@oracle40 ~]# su - flysync
Last login: Sat Nov 18 23:10:57 CST 2023 on pts/0
[flysync@oracle40 ~]$ cd /home/flysync/install
[flysync@oracle40 install]$ tar -zxf KingbaseFlySync-*-replicator.tar.gz
[flysync@oracle40 install]$ ls -l /home/flysync/install
total 139716
drwxrwxr-x 6 flysync flysync 116 Mar 20 2023 KingbaseFlySync-V002R002C004B20230320-replicator
-rw-r--r-- 1 root root 143069019 Nov 18 23:11 KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysync@oracle40 install]$
2.静默安装
a.配置数据库驱动包
[flysync@oracle40 install]$ cd /home/flysync/install/KingbaseFlySync-*-replicator/flysync-replicator/lib
[flysync@oracle40 lib]$ cp /home/flysync/install/KingbaseFlySync-*-replicator/extensions/jdbc/kingbase8-8.6.0.jar ./
[flysync@oracle40 lib]$
b.创建Kingbase FlySync安装配置文件(flysync.ini)
[flysync@oracle40 install]$ cd ~/install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini/
[flysync@oracle40 ini]$ cp flysync_oracle-logminer_source.ini ~/flysync.ini
[flysync@oracle40 ini]$
c.配置flysync.ini文件
[flysync@oracle40 ini]$ vi ~/flysync.ini
[flysync@oracle40 ini]$
[flysync@oracle40 ini]$ cat ~/flysync.ini
[defaults]
# 安装目录
install-directory=~/replicator
# 环境变量脚本位置
profile-script=~/.bash_profile
# rmi端口,默认11000
rmi-port=11000
# 服务名
[oracle]
# 监听kufl获取的端口
kufl-port=3112
# master主机名
master=oracle40
# 当前主机名
members=oracle40
# 数据源类型
datasource-type=oracle
# 同步数据库信息
replication-host=oracle40
replication-user=FLYSYNC
replication-password=flysync
replication-port=1521
# 同步服务角色
role=master
## oracle 指定连接的数据库名称
datasource-oracle-service=orcl
# oracle数据库需指定此项，解析增量方式logminer
oracle-extractor-method=redo
## oracle-LOGMINER 同步表信息、scn号差值的阈值
property=replicator.extractor.dbms.tablePatterns=FLYSYNC.*,OA11G.*
property=replicator.extractor.dbms.scnSegmentationThreshold=5000
## 过滤器开关配置
## dropstatementdata 不开启DDL同步
# svc_extractor_filters=dropstatementdata,replicate,casetransform,dropcolumn,rename
svc_extractor_filters=replicate,dropstatementdata
svc-remote-filters= rename,casetransform
## 集群参数
# property=replicator.datasource.global.connectionSpec.use_cluster=true
# property=replicator.datasource.global.connectionSpec.slave_add=192.168.28.176,192.168.28.176
# property=replicator.datasource.global.connectionSpec.slave_port=1521,1521
### 过滤器详细配置
## replicate 表过滤
property=replicator.filter.replicate.do=FLYSYNC.*,OA11G.*
# property=replicator.filter.replicate.ignore=PUBLIC.T1
## casetransform 大小写转换
property=replicator.filter.casetransform.to_upper_case=true
## 优化参数
# 达到多少条记录后，分事务，默认 500
property=replicator.extractor.dbms.maxRowsByBlock=500
# 若多少秒没有新的事务产生，那么就直接返回该事务，不再等待合并
property=replicator.extractor.dbms.maxWaitingTime=2
# 大事务优化参数
property=replicator.extractor.dbms.minRowsPerBlock=4000
# 自动恢复尝试次数，系统 OFFLINE 时，尝试将系统重新置于ONLINE状态的次数
repl_auto_recovery_max_attempts=0
# 系统 OFFLINE 时，尝试 ONLINE 之前的等待时间，s(秒)、m(分钟)、h(小时)
repl_auto_recovery_delay_interval=30s
# 系统自动恢复次数重置时间，s(秒)、m(分钟)、h(小时)
repl_auto_recovery_reset_interval=300s
[flysync@oracle40 ini]$
d.执行安装
[flysync@oracle40 ini]$ cd ~/install/KingbaseFlySync-V002R002C004B20230320-replicator/
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ ./tools/fspm install
..
WARN >> oracle40 >> ntpd is not running. It is important that configured hosts have time synchronised. (NtpdRunningCheck)
WARN >> oracle40 >> Linux swappiness is currently set to 30, on restart it will be 0, consider setting this to 10 or under to avoid swapping. (SwappinessCheck)
..License : /home/flysync/replicator/license.dat
.
#####################################################################
# Next Steps
#####################################################################
Unless automatically started, you must start the FlySync services before the
cluster will be available.
/home/flysync/replicator/flysync/cluster-home/bin/startall
Wait a minute for the services to start up and configure themselves. After
that you may proceed.
We have added FlySync environment variables to ~/.bash_profile.
Run `source ~/.bash_profile` to rebuild your environment.
Once your services start successfully replication will begin.
To look at services and perform administration, run the following command
from any database server.
/home/flysync/replicator/flysync/flysync-replicator/bin/fsrepctl services
Configuration is now complete. For further information, please consult
FlySync documentation.
NOTE >> Command successfully completed
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$
f.更换license
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ cp /install/license_26919_0_20221202_365.dat ~/replicator/license.dat
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ ls -l ~/replicator/license.dat
g.生效环境变量
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ tail -8 ~/.bash_profile
# Begin FlySync Environment for /home/flysync/replicator
# Include the FlySync variables
# Anything in this section may be changed during the next operation
if [ -f "/home/flysync/replicator/share/env.sh" ]; then
. "/home/flysync/replicator/share/env.sh"
fi
# End FlySync Environment for /home/flysync/replicator
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ source ~/.bash_profile
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$
h.配置DML映射策略(作为目标端)
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ vi ~/replicator/filters-config/rename.csv
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ cat ~/replicator/filters-config/rename.csv
a11g0017,emp0017,*,OA11G,EMP,-
oa11g0017,*,*,OA11G,-,-
flysync_kingbase8,trep_commit_seqno,*,FLYSYNC,-,-
flysync_kingbase8,consistency,*,FLYSYNC,-,-
flysync_kingbase8,heartbeat,*,FLYSYNC,-,-
flysync_kingbase8,trep_shard,*,FLYSYNC,-,-
flysync_kingbase8,trep_shard_channel,*,FLYSYNC,-,-
i.启动同步程序并置于offline 状态
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ replicator start offline
Starting FlySync Replicator Service...
Waiting for FlySync Replicator Service......
running: PID:5811
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$
j.查看同步软件的运行状态
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ fsrepctl -service oracle_source services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : master
serviceName : oracle
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$
k.配置双轨切换配置文件
[flysync@oracle40 ~]$ vi switch_conf
[flysync@oracle40 ~]$ cat switch_conf
[oracle40]
rmi_host=192.168.40.40
rmi_port=11000
[node2]
rmi_host=192.168.40.112
rmi_port=11000
[flysync@oracle40 ~]$
KES端(node2)部署同步程序
1.解压安装包并拷贝license
[root@node2 ~]# su - flysync
上一次登录：五 11月 17 22:50:12 CST 2023pts/0 上
[flysync@node2 ~]$ mkdir /home/flysync/install
[flysync@node2 ~]$ cd /home/flysync/install/
[flysync@node2 install]$ tar -zxf /install/KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz -C .
[flysync@node2 install]$ ls -l
总用量 0
drwxrwxr-x 6 flysync flysync 116 3月 20 2023 KingbaseFlySync-V002R002C004B20230320-replicator
[flysync@node2 install]$ cp /install/license.dat .
[flysync@node2 install]$
2.静默安装
1.解压安装包并拷贝license
[flysync@node2 ~]$ mkdir -p /home/flysync/install
[flysync@node2 ~]$ cd /home/flysync/install
[flysync@node2 install]$ cp /install/KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz /home/flysync/install/
[flysync@node2 install]$ tar -zxf KingbaseFlySync-*-replicator.tar.gz
[flysync@node2 install]$ ls -l /home/flysync/install
总用量 139716
drwxrwxr-x 6 flysync flysync 116 3月 20 2023 KingbaseFlySync-V002R002C004B20230320-replicator
-rw-r--r-- 1 flysync flysync 143069019 11月 19 00:24 KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysync@node2 install]$
2.静默安装
a.配置数据库驱动包
[flysync@oracle40 install]$ cd /home/flysync/install/KingbaseFlySync-*-replicator/flysync-replicator/lib
[flysync@oracle40 lib]$ cp /home/flysync/install/KingbaseFlySync-*-replicator/extensions/jdbc/kingbase8-8.6.0.jar ./
[flysync@oracle40 lib]$
b.创建Kingbase FlySync安装配置文件(flysync.ini)
[flysync@oracle40 install]$ cd ~/install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini/
[flysync@oracle40 ini]$ cp flysync_oracle-logminer_source.ini ~/flysync.ini
[flysync@oracle40 ini]$
c.配置flysync.ini文件
[flysync@node2 ini]$ cat ~/flysync.ini
[defaults]
# 安装目录
install-directory=~/replicator
# 环境变量脚本位置
profile-script=~/.bash_profile
# rmi端口,默认11000
rmi-port=11000
# 服务名
[kingbase8]
# 监听kufl获取的端口
kufl-port=3112
# master主机名
master=node2
# 当前主机名
members=node2
# 数据源类型
datasource-type=kingbase
# 同步数据库信息
replication-host=node2
replication-user=flysync
replication-password=flysync
replication-port=54321
# 同步服务角色
role=master
## 指定连接的数据库名称; 源端时，需指定解析方式
kingbase-dbname=test
kingbase-extractor-method=xlogical
## kingbase 版本号
datasource-version=8
## 过滤器开关配置
# svc_extractor_filters=replicate,casetransform,dropcolumn,rename
svc_extractor_filters=replicate,dropstatementdata
svc-remote-filters=rename,casetransform
## 集群参数
# property=replicator.datasource.global.connectionSpec.use_cluster=true
# property=replicator.datasource.global.connectionSpec.slave_add=192.168.28.176,192.168.28.176
# property=replicator.datasource.global.connectionSpec.slave_port=1521,1521
### 过滤器详细配置
## replicate 表过滤
# property=replicator.filter.replicate.do=PUBLIC.*
# property=replicator.filter.replicate.ignore=PUBLIC.T1
## casetransform 大小写转换
# property=replicator.filter.casetransform.to_upper_case=true
property=replicator.filter.replicate.do=oa11g0017.*
property=replicator.filter.casetransform.to_upper_case=false
## 优化参数
# 达到多少条记录后，分事务，默认 500
property=replicator.extractor.dbms.maxRowsByBlock=500
# 若多少秒没有新的事务产生，那么就直接返回该事务，不再等待合并
property=replicator.extractor.dbms.maxWaitingTime=2
# 大事务优化参数
property=replicator.extractor.dbms.minRowsPerBlock=4000
# 自动恢复尝试次数，系统 OFFLINE 时，尝试将系统重新置于ONLINE状态的次数
repl_auto_recovery_max_attempts=0
# 系统 OFFLINE 时，尝试 ONLINE 之前的等待时间，s(秒)、m(分钟)、h(小时)
repl_auto_recovery_delay_interval=30s
# 系统自动恢复次数重置时间，s(秒)、m(分钟)、h(小时)
repl_auto_recovery_reset_interval=300s
[flysync@node2 ini]$
d.执行安装
[flysync@node2 ini]$ cd ~/install/KingbaseFlySync-*-replicator/
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ./tools/fspm install
..
WARN >> node2 >> ntpd is not running. It is important that configured hosts have time synchronised. (NtpdRunningCheck)
WARN >> node2 >> Linux swappiness is currently set to 30, on restart it will be 0, consider setting this to 10 or under to avoid swapping. (SwappinessCheck)
....License : /home/flysync/replicator/license.dat
.
#####################################################################
# Next Steps
#####################################################################
Unless automatically started, you must start the FlySync services before the
cluster will be available.
/home/flysync/replicator/flysync/cluster-home/bin/startall
Wait a minute for the services to start up and configure themselves. After
that you may proceed.
We have added FlySync environment variables to ~/.bash_profile.
Run `source ~/.bash_profile` to rebuild your environment.
Once your services start successfully replication will begin.
To look at services and perform administration, run the following command
from any database server.
/home/flysync/replicator/flysync/flysync-replicator/bin/fsrepctl services
Configuration is now complete. For further information, please consult
FlySync documentation.
NOTE >> Command successfully completed
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$
f.更换license
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ cp /install/license_26919_0_20221202_365.dat ~/replicator/license.dat
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ls -l ~/replicator/license.dat
-rw-r--r-- 1 flysync flysync 1396 11月 19 00:33 /home/flysync/replicator/license.dat
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$
g.生效环境变量
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ tail -8 ~/.bash_profile
# Begin FlySync Environment for /home/flysync/replicator
# Include the FlySync variables
# Anything in this section may be changed during the next operation
if [ -f "/home/flysync/replicator/share/env.sh" ]; then
. "/home/flysync/replicator/share/env.sh"
fi
# End FlySync Environment for /home/flysync/replicator
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ source ~/.bash_profile
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$
h.配置DML映射策略(作为目标端)
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ vi ~/replicator/filters-config/rename.csv
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ cat ~/replicator/filters-config/rename.csv
OA11G,*,*,oa11g0017,-,-
OA11G,EMP,*,oa11g0017,emp0017,-
FLYSYNC,trep_commit_seqno,*,flysync_kingbase8,-,-
FLYSYNC,consistency,*,flysync_kingbase8,-,-
FLYSYNC,heartbeat,*,flysync_kingbase8,-,-
FLYSYNC,trep_shard,*,flysync_kingbase8,-,-
FLYSYNC,trep_shard_channel,*,flysync_kingbase8,-,-
i.启动同步程序并置于offline 状态
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ replicator start offline
Starting FlySync Replicator Service...
Waiting for FlySync Replicator Service......
running: PID:43792
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]
j.查看同步软件的运行状态
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ fsrepctl -service kingbase8 services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : master
serviceName : kingbase8
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$
k.配置双轨切换配置文件
[flysync@node2 ~]$ vi switch_conf
[flysync@node2 ~]$ cat switch_conf
[oracle40]
rmi_host=192.168.40.40
rmi_port=11000
[node2]
rmi_host=192.168.40.112
rmi_port=11000
[flysync@node2 ~]$
指定同步角色（Oracle 为主，kingbase为备)
1.ONLINE同步服务，数据开始同步
再任意节点执行
[flysync@oracle40 ~]$ repswitch -conf switch_conf show
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing show command...
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
NAME VALUE
---- -----
configName : oracle40
sourceId : oracle40
role : master
resourceVendor : oracle
state : OFFLINE:NORMAL
masterListenUri : kufl://oracle40:3112/
uptimeSeconds : 4668.738
appliedLastSeqno: -1
NAME VALUE
---- -----
configName : node2
sourceId : node2
role : master
resourceVendor : kingbase
state : OFFLINE:NORMAL
masterListenUri : kufl://node2:3112/
uptimeSeconds : 302.989
appliedLastSeqno: -1
Finish show command...
[flysync@oracle40 ~]$ repswitch -conf switch_conf offline
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing offline command...
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
oracle40 already offline
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
node2 already offline
Finish offline command...
[flysync@oracle40 ~]$ repswitch -conf switch_conf online oracle40
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
Processing online command...
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
| [ - main] INFO database.AbstractDatabase Loading database driver: oracle.jdbc.driver.OracleDriver
Set extractor start specific event ID:ora:1176447
put node oracle40 as master to online sucessful
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
put node node2 as slave to online sucessful
Finish online command...
[flysync@oracle40 ~]$ repswitch -conf switch_conf show
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing show command...
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
NAME VALUE
---- -----
configName : oracle40
sourceId : oracle40
role : master
resourceVendor : oracle
state : ONLINE
masterListenUri : kufl://oracle40:3112/
uptimeSeconds : 4703.011
appliedLastSeqno: -1
NAME VALUE
---- -----
configName : node2
sourceId : node2
role : slave
resourceVendor : kingbase
state : ONLINE
masterListenUri : kufl://node2:3112/
uptimeSeconds : 337.232
appliedLastSeqno: -1
Finish show command...
[flysync@oracle40 ~]$
2.检查同步进程状态
a.Oracle端(oracle40)
flysync@oracle40 ~]$ fsrepctl -service oracle status
Processing status command...
NAME VALUE
---- -----
appliedLastEventId : ora:1176675:1176706
appliedLastSeqno : 0
appliedLatency : 3.243
autoRecoveryEnabled : false
autoRecoveryTotal : 0
channels : 1
clusterName : oracle
currentEventId : NONE
currentTimeMillis : 1700326162042
dataServerHost : oracle40
extensions :
host : oracle40
latestEpochNumber : 0
masterConnectUri : kufl://localhost:/
masterListenUri : kufl://oracle40:3112/
maximumStoredSeqNo : 0
minimumStoredSeqNo : 0
offlineRequests : NONE
pendingError : NONE
pendingErrorCode : NONE
pendingErrorEventId : NONE
pendingErrorSeqno : -1
pendingExceptionMessage: NONE
pipelineSource : UNKNOWN
relativeLatency : 136.041
resourcePrecedence : 99
rmiPort : 11000
role : master
seqnoType : java.lang.Long
serviceName : oracle
serviceType : local
simpleServiceName : oracle
siteName : default
sourceId : oracle40
state : ONLINE
timeInStateSeconds : 135.03
timezone : GMT+08:00
transitioningTo :
uptimeSeconds : 4836.445
useSSLConnection : false
version : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@oracle40 ~]$
b.KES端(node2)
[flysync@node2 ~]$ fsrepctl -service kingbase8 status
Processing status command...
NAME VALUE
---- -----
appliedLastEventId : ora:1176675:1176706
appliedLastSeqno : 0
appliedLatency : 3.336
autoRecoveryEnabled : false
autoRecoveryTotal : 0
channels : 1
clusterName : kingbase8
currentEventId : NONE
currentTimeMillis : 1700326188743
dataServerHost : node2
extensions :
host : node2
latestEpochNumber : 0
masterConnectUri : kufl://oracle40:3112/
masterListenUri : kufl://node2:3112/
maximumStoredSeqNo : 0
minimumStoredSeqNo : 0
offlineRequests : NONE
pendingError : NONE
pendingErrorCode : NONE
pendingErrorEventId : NONE
pendingErrorSeqno : -1
pendingExceptionMessage: NONE
pipelineSource : kufl://oracle40:3112/
relativeLatency : 162.743
resourcePrecedence : 99
rmiPort : 11000
role : slave
seqnoType : java.lang.Long
serviceName : kingbase8
serviceType : local
simpleServiceName : kingbase8
siteName : default
sourceId : node2
state : ONLINE
timeInStateSeconds : 160.522
timezone : GMT+08:00
transitioningTo :
uptimeSeconds : 497.341
useSSLConnection : false
version : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@node2 ~]$
(3).KFS存量数据搬迁
1.检查replicator状态
a.Oracle端(oracle40)
[flysync@oracle40 ~]$ fsrepctl -service oracle services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: 0
appliedLatency : 3.243
role : master
serviceName : oracle
serviceType : local
started : true
state : ONLINE
Finished services command...
[flysync@oracle40 ~]$
b.KES端(node2)
[flysync@node2 ~]$ fsrepctl -service kingbase8 services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: 0
appliedLatency : 3.336
role : slave
serviceName : kingbase8
serviceType : local
started : true
state : ONLINE
Finished services command...
[flysync@node2 ~]$
2.在node2上创建oa11g0017模式
[flysync@node2 ~]$ exit
登出
[root@node2 ~]# su - kingbase
上一次登录：六 11月 18 23:04:07 CST 2023pts/0 上
[kingbase@node2 ~]$ ksql -h node2 -U flysync -d test
用户 flysync 的口令：
ksql (V8.0)
输入 "help" 来获取帮助信息.
test=# CREATE SCHEMA oa11g0017;
CREATE SCHEMA
test=#
3.用ddlscan极速模式/平滑模式（目标端流水线）迁移结构数据。
test=# \q
[kingbase@node2 ~]$ su - flysync
密码：
上一次登录：六 11月 18 23:07:27 CST 2023pts/0 上
[flysync@node2 ~]$ ddlscan -target.service kingbase8 -source.user FLYSYNC -source.pass flysync \
> -source.db orcl -source.dbtype oracle -source.host oracle40 \
> -source.port 1521 -source.schema OA11G \
> -target.db test -mgType 0 -enable-dynamic
| [ - main] INFO ddlscan.DDLScanWorker Tables or TableFile not specified - extracting everything!
| [ - main] INFO ddlscan.DDLScanWorker Connect to source database with url: jdbc:oracle:thin:@//oracle40:1521/orcl
| [ - main] INFO database.AbstractDatabase Loading database driver: oracle.jdbc.driver.OracleDriver
| [ - main] INFO database.AbstractDatabase Loading database driver: com.kingbase8.Driver
| [ - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
| [ - main] INFO filter.RenameDefinitions OA11G.*.* -> oa11g0017.-.-
| [ - main] INFO filter.RenameDefinitions OA11G.EMP.* -> oa11g0017.emp0017.-
| [ - main] INFO filter.RenameDefinitions FLYSYNC.trep_commit_seqno.* -> flysync_kingbase8.-.-
| [ - main] INFO filter.RenameDefinitions FLYSYNC.consistency.* -> flysync_kingbase8.-.-
| [ - main] INFO filter.RenameDefinitions FLYSYNC.heartbeat.* -> flysync_kingbase8.-.-
| [ - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard.* -> flysync_kingbase8.-.-
| [ - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard_channel.* -> flysync_kingbase8.-.-
| [ - main] INFO ddlscan.DDLScanWorker There is 12 tables to migration.
| [ - main] INFO ddlscan.TargetDDLScanWorker Connect to target database with url: jdbc:kingbase8://node2:54321/test
| [ - main] INFO ddlscan.TargetDDLScanWorker Drop tables on target database.
| [ - main] INFO ddlscan.DDLScanCtrl
--------------------------------------------------------------------------------
TOTAL: 12
SUCCESS: 12
FAILURE: 0
TOTAL TIME: 2s
FINISHED AT: 2023-11-19 00:58:10
OUT: /tmp/ddlscan_20231119_005807.sql
ERROR:
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------------------------------------+
|NUM |SOURCE_TABLE |TARGET_TABLE |STATUS |MESSAGE |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------------------------------------+
|1 |OA11G.COUNTRIES |oa11g0017.countries |Success | |
|2 |OA11G.DEPARTMENTS |oa11g0017.departments |Success | |
|3 |OA11G.DEPT |oa11g0017.dept |Success | |
|4 |OA11G.EMP |oa11g0017.emp0017 |Success | |
|5 |OA11G.EMPLOYEES |oa11g0017.employees |Success | |
|6 |OA11G.JOBS |oa11g0017.jobs |Success | |
|7 |OA11G.JOB_HISTORY |oa11g0017.job_history |Success | |
|8 |OA11G.LOCATIONS |oa11g0017.locations |Success | |
|9 |OA11G.MLOG$_EMP |oa11g0017.mlog$_emp |Success | |
|10 |OA11G.REGIONS |oa11g0017.regions |Success | |
|11 |OA11G.RUPD$_EMP |oa11g0017.rupd$_emp |Success | |
|12 |OA11G.SALGRADE |oa11g0017.salgrade |Success | |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------------------------------------+
[flysync@node2 ~]$ exit
登出
[kingbase@node2 ~]$ su – kingbase
su: user – does not exist
[kingbase@node2 ~]$ ksql -h node2 -U flysync -d test
用户 flysync 的口令：
ksql (V8.0)
输入 "help" 来获取帮助信息.
test=# \dt+ oa11g0017.*;
关联列表
架构模式 | 名称 | 类型 | 拥有者 | 大小 | 描述
-----------+-------------+--------+---------+------------+------
oa11g0017 | countries | 数据表 | flysync | 8192 bytes |
oa11g0017 | departments | 数据表 | flysync | 0 bytes |
oa11g0017 | dept | 数据表 | flysync | 0 bytes |
oa11g0017 | emp0017 | 数据表 | flysync | 0 bytes |
oa11g0017 | employees | 数据表 | flysync | 0 bytes |
oa11g0017 | job_history | 数据表 | flysync | 0 bytes |
oa11g0017 | jobs | 数据表 | flysync | 0 bytes |
oa11g0017 | locations | 数据表 | flysync | 0 bytes |
oa11g0017 | mlog$_emp | 数据表 | flysync | 8192 bytes |
oa11g0017 | regions | 数据表 | flysync | 8192 bytes |
oa11g0017 | rupd$_emp | 数据表 | flysync | 8192 bytes |
oa11g0017 | salgrade | 数据表 | flysync | 8192 bytes |
(12 行记录)
test=#
4.用loader极速模式进行数据搬迁
在oracle40节点上执行
[flysync@oracle40 ~]$ loader -source.service oracle -source.user FLYSYNC -source.pass flysync \
> -source.db orcl -source.dbtype oracle -source.host oracle40 \
> -source.port 1521 -source.schema OA11G \
> -source.rmiHost oracle40 -source.rmiPort 11000 \
> -target.rmiHost node2 -target.rmiPort 11000 \
> -target.service kingbase8 -mgType 2 -clean
| [ - main] INFO database.AbstractDatabase Loading database driver: oracle.jdbc.driver.OracleDriver
| [ - main] INFO loader.LoaderWorker Tables not specified - extracting everything!
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
| [ - main] INFO loader.SourceSeamlessLoaderWorker Source service status GOING-OFFLINE
| [ - main] INFO database.OracleDatabase lock table "OA11G"."COUNTRIES","OA11G"."DEPARTMENTS","OA11G"."DEPT","OA11G"."EMP","OA11G"."EMPLOYEES","OA11G"."JOBS","OA11G"."JOB_HISTORY","OA11G"."LOCATIONS","OA11G"."MLOG$_EMP","OA11G"."REGIONS","OA11G"."RUPD$_EMP","OA11G"."SALGRADE" in exclusive mode
| [ - main] INFO loader.SourceSeamlessLoaderWorker Waiting for state OFFLINE:NORMAL
| [ - main] INFO loader.SourceSeamlessLoaderWorker Source service status GOING-ONLINE_TO_EVENT_ID at event id ora:1177061
| [ - main] INFO loader.SourceSeamlessLoaderWorker Waiting for state OFFLINE:NORMAL
| [ - main] INFO loader.LoaderWorker select "eventid" from "FLYSYNC"."trep_commit_seqno"
| [ - main] INFO loader.LoaderWorker Import table OA11G.COUNTRIES
| [ - main] INFO loader.LoaderWorker Import table OA11G.DEPARTMENTS
| [ - main] INFO loader.LoaderWorker Import table OA11G.DEPT
| [ - main] INFO loader.LoaderWorker Import table OA11G.EMP
| [ - main] INFO loader.LoaderWorker Import table OA11G.EMPLOYEES
| [ - main] INFO loader.LoaderWorker Import table OA11G.JOBS
| [ - main] INFO loader.LoaderWorker Import table OA11G.JOB_HISTORY
| [ - main] INFO loader.LoaderWorker Import table OA11G.LOCATIONS
| [ - main] INFO loader.LoaderWorker Import table OA11G.MLOG$_EMP
| [ - main] INFO loader.LoaderWorker Import table OA11G.REGIONS
| [ - main] INFO loader.LoaderWorker Import table OA11G.RUPD$_EMP
| [ - main] INFO loader.LoaderWorker Import table OA11G.SALGRADE
| [ - main] INFO conf.ReplicatorRuntime Replicator role: master
| [oracle - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true
| [oracle - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
| [oracle - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
| [oracle - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [oracle - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [oracle - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
| [oracle - main] INFO pipeline.Pipeline Configuring pipeline: master
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=global
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=extractor
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=oracle_cdc
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=oracle_cdc className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=oracle_logminer
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=oracle_logminer className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=oracle_redo
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=oracle_redo className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=file_metadata
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
| [oracle - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
| [oracle - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false
| [oracle - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is disabled; extractor will not reposition automatically
| [oracle - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-kufl
| [oracle - main] INFO conf.ReplicatorRuntime Preparing pipeline for use: master
| [oracle - main] INFO pipeline.Pipeline Preparing pipeline: master
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=file_metadata
| [oracle - main] INFO datasource.FileDataSource Initializing data source files: service=oracle directory=/home/flysync/replicator/metadata/applier
| [oracle - main] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=global
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=oracle_cdc
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=oracle_logminer
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=oracle_redo
| [oracle - main] INFO kufl.KUFL Connecting to data source
| [oracle - pool-2-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: master
| [oracle - main] INFO kufl.log.DiskLog Using directory '/home/flysync/replicator/kufl/oracle/' for replicator logs
| [oracle - main] INFO kufl.log.DiskLog Checksums enabled for log records: true
| [oracle - main] INFO kufl.log.DiskLog Acquired write lock; log is writable
| [oracle - main] INFO kufl.log.DiskLog Loaded event serializer class: com.kingbase.flysync.replicator.kufl.serializer.ProtobufSerializer
| [oracle - main] INFO kufl.log.LogIndex Building file index on log directory: /home/flysync/replicator/kufl/oracle
| [oracle - main] INFO kufl.log.LogIndex Constructed index; total log files added=1
| [oracle - main] INFO kufl.log.DiskLog Validating last log file: /home/flysync/replicator/kufl/oracle/kufl.data.0000000001
| [oracle - main] INFO kufl.log.DiskLog Setting up log flush policy: fsyncIntervalMillis=0 fsyncOnFlush=false
| [oracle - main] INFO kufl.log.DiskLog Idle log connection timeout: 28800000ms
| [oracle - main] INFO kufl.log.DiskLog Log preparation is complete
| [oracle - main] INFO kufl.KUFL Log preparation is complete
| [oracle - main] INFO kufl.KUFL Restart consistency checking skipped as we are not recovering a master to a slave
| [oracle - main] INFO kufl.Server Opening KUFL server: store name=kufl host=0.0.0.0 port=3112
| [oracle - main] INFO kufl.Server Opened socket: host=/0.0.0.0:3112 port=3112 useSSL=false
| [oracle - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-kufl
| [oracle - main] INFO kufl.KUFLStoreApplier Storage adapter is prepared: name=kufl
| [oracle - main] INFO pipeline.StageTaskGroup [main-q-to-kufl-0] Setting extractor position: seqno=1 event=ora:1177078:1177095
| [oracle - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q
| [oracle - main] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter
| [oracle - main] INFO pipeline.StageTaskGroup [main-loader-to-q-0] Setting extractor position: seqno=1 event=ora:1177078:1177095
| [oracle - main] INFO pipeline.Pipeline Starting pipeline: master
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=14 eventid=ora:1177078:1177095
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 13
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=14 eventid=ora:1177078:1177095
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Task event count: 13
| [oracle - pool-2-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: master
| [oracle - main] INFO pipeline.Pipeline Shutting down pipeline: master
| [oracle - main] INFO pipeline.Pipeline Releasing pipeline: master
| [oracle - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q
| [oracle - main] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter
| [oracle - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-kufl
| [oracle - main] INFO kufl.Server Stopping server thread
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server Server thread cancelled
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server Closing connector handlers for KUFL Server: store=kufl
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server Closing socket: store=kufl host=/0.0.0.0:3112 port=3112
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server KUFL thread done: store=kufl
| [oracle - main] INFO conf.ReplicatorRuntime Replicator role: master
| [oracle - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true
| [oracle - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
| [oracle - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
| [oracle - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [oracle - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [oracle - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
| [oracle - main] INFO pipeline.Pipeline Configuring pipeline: master
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=global
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=extractor
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=oracle_cdc
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=oracle_cdc className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=oracle_logminer
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=oracle_logminer className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=oracle_redo
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=oracle_redo className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=file_metadata
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
| [oracle - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
| [oracle - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false
| [oracle - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is disabled; extractor will not reposition automatically
| [oracle - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-kufl
| [oracle - main] INFO conf.ReplicatorRuntime Replicator role: master
| [oracle - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true
| [oracle - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
| [oracle - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
| [oracle - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [oracle - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [oracle - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
| [oracle - main] INFO pipeline.Pipeline Configuring pipeline: master
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=global
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=extractor
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=oracle_cdc
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=oracle_cdc className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=oracle_logminer
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=oracle_logminer className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=oracle_redo
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=oracle_redo className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=file_metadata
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
| [oracle - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
| [oracle - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false
| [oracle - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is disabled; extractor will not reposition automatically
| [oracle - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-kufl
| [oracle - main] INFO conf.ReplicatorRuntime Preparing pipeline for use: master
| [oracle - main] INFO pipeline.Pipeline Preparing pipeline: master
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=file_metadata
| [oracle - main] INFO datasource.FileDataSource Initializing data source files: service=oracle directory=/home/flysync/replicator/metadata/applier
| [oracle - main] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=global
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=oracle_cdc
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=oracle_logminer
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=oracle_redo
| [oracle - main] INFO kufl.KUFL Connecting to data source
| [oracle - pool-5-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: master
| [oracle - main] INFO kufl.log.DiskLog Using directory '/home/flysync/replicator/kufl/oracle/' for replicator logs
| [oracle - main] INFO kufl.log.DiskLog Checksums enabled for log records: true
| [oracle - main] INFO kufl.log.DiskLog Acquired write lock; log is writable
| [oracle - main] INFO kufl.log.DiskLog Loaded event serializer class: com.kingbase.flysync.replicator.kufl.serializer.ProtobufSerializer
| [oracle - main] INFO kufl.log.LogIndex Building file index on log directory: /home/flysync/replicator/kufl/oracle
| [oracle - main] INFO kufl.log.LogIndex Constructed index; total log files added=1
| [oracle - main] INFO kufl.log.DiskLog Validating last log file: /home/flysync/replicator/kufl/oracle/kufl.data.0000000001
| [oracle - main] INFO kufl.log.DiskLog Setting up log flush policy: fsyncIntervalMillis=0 fsyncOnFlush=false
| [oracle - main] INFO kufl.log.DiskLog Idle log connection timeout: 28800000ms
| [oracle - main] INFO kufl.log.DiskLog Log preparation is complete
| [oracle - main] INFO kufl.KUFL Log preparation is complete
| [oracle - main] INFO kufl.KUFL Restart consistency checking skipped as we are not recovering a master to a slave
| [oracle - main] INFO kufl.Server Opening KUFL server: store name=kufl host=0.0.0.0 port=3112
| [oracle - main] INFO kufl.Server Opened socket: host=/0.0.0.0:3112 port=3112 useSSL=false
| [oracle - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-kufl
| [oracle - main] INFO kufl.KUFLStoreApplier Storage adapter is prepared: name=kufl
| [oracle - main] INFO pipeline.StageTaskGroup [main-q-to-kufl-0] Setting extractor position: seqno=14 event=ora:1177078:1177095
| [oracle - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q
| [oracle - main] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter
| [oracle - main] INFO pipeline.StageTaskGroup [main-loader-to-q-0] Setting extractor position: seqno=14 event=ora:1177078:1177095
| [oracle - main] INFO pipeline.Pipeline Starting pipeline: master
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=27 eventid=ora:1177078:1177095
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 13
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=27 eventid=ora:1177078:1177095
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Task event count: 13
| [oracle - pool-5-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: master
| [oracle - main] INFO pipeline.Pipeline Shutting down pipeline: master
| [oracle - main] INFO pipeline.Pipeline Releasing pipeline: master
| [oracle - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q
| [oracle - main] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter
| [oracle - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-kufl
| [oracle - main] INFO kufl.Server Stopping server thread
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server Server thread cancelled
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server Closing connector handlers for KUFL Server: store=kufl
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server Closing socket: store=kufl host=/0.0.0.0:3112 port=3112
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server KUFL thread done: store=kufl
| [oracle - pipeline-0] INFO conf.ReplicatorRuntime Replicator role: master
| [oracle - pipeline-0] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true
| [oracle - pipeline-0] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
| [oracle - pipeline-0] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
| [oracle - pipeline-0] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [oracle - pipeline-0] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [oracle - pipeline-0] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
| [oracle - pipeline-0] INFO pipeline.Pipeline Configuring pipeline: master
| [oracle - pipeline-0] INFO datasource.DataSourceService Configuring data source: name=global
| [oracle - pipeline-0] INFO datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - pipeline-0] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - pipeline-0] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - pipeline-0] INFO datasource.DataSourceService Configuring data source: name=extractor
| [oracle - pipeline-0] INFO datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
| [oracle - pipeline-0] INFO datasource.DataSourceService Configuring data source: name=oracle_cdc
| [oracle - pipeline-0] INFO datasource.DataSourceManager Loading data source: name=oracle_cdc className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - pipeline-0] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - pipeline-0] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - pipeline-0] INFO datasource.DataSourceService Configuring data source: name=oracle_logminer
| [oracle - pipeline-0] INFO datasource.DataSourceManager Loading data source: name=oracle_logminer className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - pipeline-0] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - pipeline-0] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - pipeline-0] INFO datasource.DataSourceService Configuring data source: name=oracle_redo
| [oracle - pipeline-0] INFO datasource.DataSourceManager Loading data source: name=oracle_redo className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - pipeline-0] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - pipeline-0] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - pipeline-0] INFO datasource.DataSourceService Configuring data source: name=file_metadata
| [oracle - pipeline-0] INFO datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
| [oracle - pipeline-0] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - pipeline-0] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - pipeline-0] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
| [oracle - pipeline-0] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
| [oracle - pipeline-0] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false
| [oracle - pipeline-0] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is disabled; extractor will not reposition automatically
| [oracle - pipeline-0] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-kufl
| [oracle - pipeline-0] INFO conf.ReplicatorRuntime Preparing pipeline for use: master
| [oracle - pipeline-0] INFO pipeline.Pipeline Preparing pipeline: master
| [oracle - pipeline-0] INFO datasource.DataSourceService Preparing and initializing data source: name=file_metadata
| [oracle - pipeline-0] INFO datasource.FileDataSource Initializing data source files: service=oracle directory=/home/flysync/replicator/metadata/applier
| [oracle - pipeline-0] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1
| [oracle - pipeline-0] INFO datasource.DataSourceService Preparing and initializing data source: name=global
| [oracle - pipeline-0] INFO datasource.DataSourceService Preparing and initializing data source: name=oracle_cdc
| [oracle - pipeline-0] INFO datasource.DataSourceService Preparing and initializing data source: name=oracle_logminer
| [oracle - pipeline-0] INFO datasource.DataSourceService Preparing and initializing data source: name=oracle_redo
| [oracle - pipeline-0] INFO kufl.KUFL Connecting to data source
| [oracle - pool-7-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: master
| [oracle - pipeline-0] INFO kufl.log.DiskLog Using directory '/home/flysync/replicator/kufl/oracle/' for replicator logs
| [oracle - pipeline-0] INFO kufl.log.DiskLog Checksums enabled for log records: true
| [oracle - pipeline-0] INFO kufl.log.DiskLog Acquired write lock; log is writable
| [oracle - pipeline-0] INFO kufl.log.DiskLog Loaded event serializer class: com.kingbase.flysync.replicator.kufl.serializer.ProtobufSerializer
| [oracle - pipeline-0] INFO kufl.log.LogIndex Building file index on log directory: /home/flysync/replicator/kufl/oracle
| [oracle - pipeline-0] INFO kufl.log.LogIndex Constructed index; total log files added=1
| [oracle - pipeline-0] INFO kufl.log.DiskLog Validating last log file: /home/flysync/replicator/kufl/oracle/kufl.data.0000000001
| [oracle - pipeline-0] INFO kufl.log.DiskLog Setting up log flush policy: fsyncIntervalMillis=0 fsyncOnFlush=false
| [oracle - pipeline-0] INFO kufl.log.DiskLog Idle log connection timeout: 28800000ms
| [oracle - pipeline-0] INFO kufl.log.DiskLog Log preparation is complete
| [oracle - pipeline-0] INFO kufl.KUFL Log preparation is complete
| [oracle - pipeline-0] INFO kufl.KUFL Restart consistency checking skipped as we are not recovering a master to a slave
| [oracle - pipeline-0] INFO kufl.Server Opening KUFL server: store name=kufl host=0.0.0.0 port=3112
| [oracle - pipeline-0] INFO kufl.Server Opened socket: host=/0.0.0.0:3112 port=3112 useSSL=false
| [oracle - pipeline-0] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-kufl
| [oracle - pipeline-0] INFO kufl.KUFLStoreApplier Storage adapter is prepared: name=kufl
| [oracle - pipeline-0] INFO pipeline.StageTaskGroup [pipeline-0-q-to-kufl-0] Setting extractor position: seqno=27 event=ora:1177078:1177095
| [oracle - pipeline-0] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q
| [oracle - pipeline-0] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter
| [oracle - pipeline-0] INFO pipeline.StageTaskGroup [pipeline-0-loader-to-q-0] Setting extractor position: seqno=27 event=ora:1177078:1177095
| [oracle - pipeline-0] INFO pipeline.Pipeline Starting pipeline: master
| [oracle - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [oracle - pipeline-0-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [oracle - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "OA11G"."COUNTRIES":0-25
| [oracle - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "OA11G"."DEPARTMENTS":0-27
| [oracle - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "OA11G"."DEPT":0-4
| [oracle - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "OA11G"."EMP":0-14
| [oracle - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "OA11G"."EMPLOYEES":0-107
| [oracle - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "OA11G"."JOBS":0-19
| [oracle - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "OA11G"."JOB_HISTORY":0-10
| [oracle - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "OA11G"."LOCATIONS":0-23
| [oracle - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "OA11G"."MLOG$_EMP":0-0
| [oracle - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "OA11G"."REGIONS":0-4
| [oracle - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "OA11G"."RUPD$_EMP":0-0
| [oracle - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "OA11G"."SALGRADE":0-5
| [oracle - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [oracle - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [oracle - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=40 eventid=ora:1177078:1177095
| [oracle - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 13
| [oracle - pipeline-0-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [oracle - pipeline-0-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [oracle - pipeline-0-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=40 eventid=ora:1177078:1177095
| [oracle - pipeline-0-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Task event count: 13
| [oracle - pool-7-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: master
| [oracle - pipeline-0] INFO pipeline.Pipeline Shutting down pipeline: master
| [oracle - pipeline-0] INFO pipeline.Pipeline Releasing pipeline: master
| [oracle - pipeline-0] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q
| [oracle - pipeline-0] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter
| [oracle - pipeline-0] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-kufl
| [oracle - pipeline-0] INFO kufl.Server Stopping server thread
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server Server thread cancelled
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server Closing connector handlers for KUFL Server: store=kufl
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server Closing socket: store=kufl host=/0.0.0.0:3112 port=3112
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server KUFL thread done: store=kufl
| [oracle - main] INFO loader.SourceSeamlessLoaderWorker Tables imported
| [oracle - main] INFO loader.LoaderWorker | -------------------------打印执行结果开始-------------------------
| [oracle - main] INFO loader.LoaderWorker | OA11G.COUNTRIES : success
| [oracle - main] INFO loader.LoaderWorker | OA11G.JOB_HISTORY : success
| [oracle - main] INFO loader.LoaderWorker | OA11G.MLOG$_EMP : success
| [oracle - main] INFO loader.LoaderWorker | OA11G.DEPT : success
| [oracle - main] INFO loader.LoaderWorker | OA11G.REGIONS : success
| [oracle - main] INFO loader.LoaderWorker | OA11G.DEPARTMENTS : success
| [oracle - main] INFO loader.LoaderWorker | OA11G.EMPLOYEES : success
| [oracle - main] INFO loader.LoaderWorker | OA11G.SALGRADE : success
| [oracle - main] INFO loader.LoaderWorker | OA11G.LOCATIONS : success
| [oracle - main] INFO loader.LoaderWorker | OA11G.EMP : success
| [oracle - main] INFO loader.LoaderWorker | OA11G.JOBS : success
| [oracle - main] INFO loader.LoaderWorker | OA11G.RUPD$_EMP : success
| [oracle - main] INFO loader.LoaderWorker | -------------------------打印执行结果结束-------------------------
| [oracle - main] INFO loader.SourceSeamlessLoaderWorker Data loading to the relocation channel is completed.
| [oracle - main] INFO conf.ReplicatorRuntime Replicator role: master
| [oracle - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true
| [oracle - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
| [oracle - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
| [oracle - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [oracle - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [oracle - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
| [oracle - main] INFO pipeline.Pipeline Configuring pipeline: master
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=global
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=extractor
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=oracle_cdc
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=oracle_cdc className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=oracle_logminer
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=oracle_logminer className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=oracle_redo
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=oracle_redo className=com.kingbase.flysync.replicator.datasource.SqlDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO datasource.DataSourceService Configuring data source: name=file_metadata
| [oracle - main] INFO datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
| [oracle - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [oracle - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [oracle - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
| [oracle - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
| [oracle - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false
| [oracle - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is disabled; extractor will not reposition automatically
| [oracle - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-kufl
| [oracle - main] INFO conf.ReplicatorRuntime Preparing pipeline for use: master
| [oracle - main] INFO pipeline.Pipeline Preparing pipeline: master
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=file_metadata
| [oracle - main] INFO datasource.FileDataSource Initializing data source files: service=oracle directory=/home/flysync/replicator/metadata/applier
| [oracle - main] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=global
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=oracle_cdc
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=oracle_logminer
| [oracle - main] INFO datasource.DataSourceService Preparing and initializing data source: name=oracle_redo
| [oracle - main] INFO kufl.KUFL Connecting to data source
| [oracle - pool-9-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: master
| [oracle - main] INFO kufl.log.DiskLog Using directory '/home/flysync/replicator/kufl/oracle/' for replicator logs
| [oracle - main] INFO kufl.log.DiskLog Checksums enabled for log records: true
| [oracle - main] INFO kufl.log.DiskLog Acquired write lock; log is writable
| [oracle - main] INFO kufl.log.DiskLog Loaded event serializer class: com.kingbase.flysync.replicator.kufl.serializer.ProtobufSerializer
| [oracle - main] INFO kufl.log.LogIndex Building file index on log directory: /home/flysync/replicator/kufl/oracle
| [oracle - main] INFO kufl.log.LogIndex Constructed index; total log files added=1
| [oracle - main] INFO kufl.log.DiskLog Validating last log file: /home/flysync/replicator/kufl/oracle/kufl.data.0000000001
| [oracle - main] INFO kufl.log.DiskLog Setting up log flush policy: fsyncIntervalMillis=0 fsyncOnFlush=false
| [oracle - main] INFO kufl.log.DiskLog Idle log connection timeout: 28800000ms
| [oracle - main] INFO kufl.log.DiskLog Log preparation is complete
| [oracle - main] INFO kufl.KUFL Log preparation is complete
| [oracle - main] INFO kufl.KUFL Restart consistency checking skipped as we are not recovering a master to a slave
| [oracle - main] INFO kufl.Server Opening KUFL server: store name=kufl host=0.0.0.0 port=3112
| [oracle - main] INFO kufl.Server Opened socket: host=/0.0.0.0:3112 port=3112 useSSL=false
| [oracle - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-kufl
| [oracle - main] INFO kufl.KUFLStoreApplier Storage adapter is prepared: name=kufl
| [oracle - main] INFO pipeline.StageTaskGroup [main-q-to-kufl-0] Setting extractor position: seqno=40 event=ora:1177078:1177095
| [oracle - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q
| [oracle - main] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter
| [oracle - main] INFO pipeline.StageTaskGroup [main-loader-to-q-0] Setting extractor position: seqno=40 event=ora:1177078:1177095
| [oracle - main] INFO pipeline.Pipeline Starting pipeline: master
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=53 eventid=ora:1177078:1177095
| [oracle - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 13
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=53 eventid=ora:1177078:1177095
| [oracle - main-q-to-kufl-0] INFO pipeline.SingleThreadStageTask Task event count: 13
| [oracle - pool-9-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: master
| [oracle - main] INFO pipeline.Pipeline Shutting down pipeline: master
| [oracle - main] INFO pipeline.Pipeline Releasing pipeline: master
| [oracle - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q
| [oracle - main] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter
| [oracle - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-kufl
| [oracle - main] INFO kufl.Server Stopping server thread
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server Server thread cancelled
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server Closing connector handlers for KUFL Server: store=kufl
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server Closing socket: store=kufl host=/0.0.0.0:3112 port=3112
| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO kufl.Server KUFL thread done: store=kufl
| [oracle - main] INFO loader.SourceSeamlessLoaderWorker Source service status GOING-ONLINE
| [oracle - main] INFO loader.LoaderCtrl Total time:7
| [oracle - main] INFO loader.LoaderCtrl
--------------------------------------------------------------------------------
TOTAL: 12
SUCCESS: 12
FAILURE: 0
TOTAL TIME: 7s
FINISHED AT: 2023-11-19 01:00:08
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------+
|NUM |SOURCE_TABLE |TARGET_TABLE |STATUS |TOTAL |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------+
|1 |OA11G.DEPARTMENTS | |Success |27 |
|2 |OA11G.EMPLOYEES | |Success |107 |
|3 |OA11G.RUPD$_EMP | |Success |0 |
|4 |OA11G.REGIONS | |Success |4 |
|5 |OA11G.LOCATIONS | |Success |23 |
|6 |OA11G.JOB_HISTORY | |Success |10 |
|7 |OA11G.JOBS | |Success |19 |
|8 |OA11G.EMP | |Success |14 |
|9 |OA11G.MLOG$_EMP | |Success |0 |
|10 |OA11G.COUNTRIES | |Success |25 |
|11 |OA11G.DEPT | |Success |4 |
|12 |OA11G.SALGRADE | |Success |5 |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------+
[flysync@oracle40 ~]$
在node2节点上执行：查看数据是否迁移
test=# \dt+ oa11g0017.*;
关联列表
架构模式 | 名称 | 类型 | 拥有者 | 大小 | 描述
-----------+-------------+--------+---------+------------+------
oa11g0017 | countries | 数据表 | flysync | 16 kB |
oa11g0017 | departments | 数据表 | flysync | 8192 bytes |
oa11g0017 | dept | 数据表 | flysync | 8192 bytes |
oa11g0017 | emp0017 | 数据表 | flysync | 8192 bytes |
oa11g0017 | employees | 数据表 | flysync | 40 kB |
oa11g0017 | job_history | 数据表 | flysync | 8192 bytes |
oa11g0017 | jobs | 数据表 | flysync | 8192 bytes |
oa11g0017 | locations | 数据表 | flysync | 8192 bytes |
oa11g0017 | mlog$_emp | 数据表 | flysync | 8192 bytes |
oa11g0017 | regions | 数据表 | flysync | 16 kB |
oa11g0017 | rupd$_emp | 数据表 | flysync | 8192 bytes |
oa11g0017 | salgrade | 数据表 | flysync | 16 kB |
(12 行记录)
test=# SELECT count(*) FROM oa11g0017.emp0017;
count
-------
14
(1 行记录)
test=#
(4).增量数据同步(Oracle为主，Kingbase 为备)
1.指定主库和备库，ONLINE同步服务，数据开始同步
指定主库和备库，ONLINE同步服务，数据开始同步
[flysync@oracle40 ~]$ repswitch -conf switch_conf show
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing show command...
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
NAME VALUE
---- -----
configName : oracle40
sourceId : oracle40
role : master
resourceVendor : oracle
state : ONLINE
masterListenUri : kufl://oracle40:3112/
uptimeSeconds : 5812.324
appliedLastSeqno: 56
NAME VALUE
---- -----
configName : node2
sourceId : node2
role : slave
resourceVendor : kingbase
state : ONLINE
masterListenUri : kufl://node2:3112/
uptimeSeconds : 1446.555
appliedLastSeqno: 56
Finish show command...
[flysync@oracle40 ~]$ repswitch -conf switch_conf offline
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing offline command...
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
oracle40 offline sucessful
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
node2 offline sucessful
Finish offline command...
[flysync@oracle40 ~]$ repswitch -conf switch_conf online oracle40
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
Processing online command...
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
| [ - main] INFO database.AbstractDatabase Loading database driver: oracle.jdbc.driver.OracleDriver
Set extractor start specific event ID:ora:1177702
put node oracle40 as master to online sucessful
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
put node node2 as slave to online sucessful
Finish online command...
[flysync@oracle40 ~]$ repswitch -conf switch_conf show
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing show command...
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
NAME VALUE
---- -----
configName : oracle40
sourceId : oracle40
role : master
resourceVendor : oracle
state : ONLINE
masterListenUri : kufl://oracle40:3112/
uptimeSeconds : 5815.605
appliedLastSeqno: 56
NAME VALUE
---- -----
configName : node2
sourceId : node2
role : slave
resourceVendor : kingbase
state : ONLINE
masterListenUri : kufl://node2:3112/
uptimeSeconds : 1449.821
appliedLastSeqno: 56
Finish show command...
[flysync@oracle40 ~]$
2.查看同步进程状态
a.Oracle端(oracle40)
[flysync@oracle40 ~]$ fsrepctl -service oracle status
Processing status command...
NAME VALUE
---- -----
appliedLastEventId : ora:1177724:1177741
appliedLastSeqno : 57
appliedLatency : 2.361
autoRecoveryEnabled : false
autoRecoveryTotal : 0
channels : 1
clusterName : oracle
currentEventId : NONE
currentTimeMillis : 1700327213363
dataServerHost : oracle40
extensions :
host : oracle40
latestEpochNumber : 57
masterConnectUri : kufl://localhost:/
masterListenUri : kufl://oracle40:3112/
maximumStoredSeqNo : 57
minimumStoredSeqNo : 0
offlineRequests : NONE
pendingError : NONE
pendingErrorCode : NONE
pendingErrorEventId : NONE
pendingErrorSeqno : -1
pendingExceptionMessage: NONE
pipelineSource : UNKNOWN
relativeLatency : 73.363
resourcePrecedence : 99
rmiPort : 11000
role : master
seqnoType : java.lang.Long
serviceName : oracle
serviceType : local
simpleServiceName : oracle
siteName : default
sourceId : oracle40
state : ONLINE
timeInStateSeconds : 72.904
timezone : GMT+08:00
transitioningTo :
uptimeSeconds : 5887.766
useSSLConnection : false
version : Kingbase FlySync V002R002C004B20230320
Finished status comma
b.KES端(node2)
test=# \q
[kingbase@node2 ~]$ exit
登出
[root@node2 ~]# su - flysync
上一次登录：日 11月 19 00:57:56 CST 2023pts/0 上
[flysync@node2 ~]$ fsrepctl -service kingbase8 status
Processing status command...
NAME VALUE
---- -----
appliedLastEventId : ora:1177724:1177741
appliedLastSeqno : 57
appliedLatency : 2.398
autoRecoveryEnabled : false
autoRecoveryTotal : 0
channels : 1
clusterName : kingbase8
currentEventId : NONE
currentTimeMillis : 1700327261503
dataServerHost : node2
extensions :
host : node2
latestEpochNumber : 57
masterConnectUri : kufl://oracle40:3112/
masterListenUri : kufl://node2:3112/
maximumStoredSeqNo : 57
minimumStoredSeqNo : 0
offlineRequests : NONE
pendingError : NONE
pendingErrorCode : NONE
pendingErrorEventId : NONE
pendingErrorSeqno : -1
pendingExceptionMessage: NONE
pipelineSource : kufl://oracle40:3112/
relativeLatency : 121.503
resourcePrecedence : 99
rmiPort : 11000
role : slave
seqnoType : java.lang.Long
serviceName : kingbase8
serviceType : local
simpleServiceName : kingbase8
siteName : default
sourceId : node2
state : ONLINE
timeInStateSeconds : 120.779
timezone : GMT+08:00
transitioningTo :
uptimeSeconds : 1570.101
useSSLConnection : false
version : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@node2 ~]$
3.数据同步验证(验证DML)
a.在源端（oracle40）生成数据
[flysync@oracle40 ~]$ exit
logout
[root@oracle40 ~]# su - oracle
Last login: Sat Nov 18 22:51:46 CST 2023 on pts/0
[oracle@oracle40 ~]$ sqlplus OA11G/oa11g
SQL*Plus: Release 11.2.0.4.0 Production on Sun Nov 19 01:10:46 2023
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> DESC oa11g.emp;
DESC oa11g.dept;
INSERT INTO oa11g.emp(EMPNO, ENAME) VALUES(7935,'KFS');
INSERT INTO oa11g.dept VALUES(50,'engineer','NEW YORK');
COMMIT;
Name Null? Type
----------------------------------------- -------- ----------------------------
EMPNO NOT NULL NUMBER(4)
ENAME VARCHAR2(10)
JOB VARCHAR2(9)
MGR NUMBER(4)
HIREDATE DATE
SAL NUMBER(7,2)
COMM NUMBER(7,2)
DEPTNO NUMBER(2)
SQL> Name Null? Type
----------------------------------------- -------- ----------------------------
DEPTNO NOT NULL NUMBER(2)
DNAME VARCHAR2(14)
LOC VARCHAR2(13)
SQL>
1 row created.
SQL>
1 row created.
SQL>
Commit complete.
SQL>
b.查看源端（oracle40）kufl 是否解析
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
[oracle@oracle40 ~]$ exit
logout
[root@oracle40 ~]# su - flysync
Last login: Sat Nov 18 23:11:24 CST 2023 on pts/0
[flysync@oracle40 ~]$ kufl list -last 1
SEQ# = 58 / FRAG# = 0 (last frag)
- TIME = 2023-11-19 01:10:58.0
- EPOCH# = 57
- EVENTID = ora:1178165:1178299
- SOURCEID = oracle40
- METADATA = [dbms_type=oracle;tz_aware=true;xid=1688965824382101;tz_aware=true;all_tables=[OA11G.EMP, OA11G.MLOG$_EMP, OA11G.DEPT];service=oracle;shard=OA11G]
- TYPE = com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS = []
- SQL(0) =
- ACTION = INSERT
- SCHEMA = OA11G
- TABLE = EMP
- ROW# = 0
- COL(1: EMPNO) = 7935
- COL(2: ENAME) = KFS
- COL(3: JOB) = NULL
- COL(4: MGR) = NULL
- COL(5: HIREDATE) = NULL
- COL(6: SAL) = NULL
- COL(7: COMM) = NULL
- COL(8: DEPTNO) = NULL
- SQL(1) =
- ACTION = INSERT
- SCHEMA = OA11G
- TABLE = MLOG$_EMP
- ROW# = 0
- COL(1: EMPNO) = 7935
- COL(2: SNAPTIME$$) = 4000-01-01 00:00:00
- COL(3: DMLTYPE$$) = I
- COL(4: OLD_NEW$$) = N
- COL(5: CHANGE_VECTOR$$) = [B@2c6a3f77
- COL(6: XID$$) = 1688965824382101
- SQL(2) =
- ACTION = INSERT
- SCHEMA = OA11G
- TABLE = DEPT
- ROW# = 0
- COL(1: DEPTNO) = 50
- COL(2: DNAME) = engineer
- COL(3: LOC) = NEW YORK
- SQL(3) =
- ACTION = DELETE
- SCHEMA = OA11G
- TABLE = MLOG$_EMP
- ROW# = 0
- KEY(1: EMPNO) = 7935
- KEY(2: SNAPTIME$$) = 4000-01-01 00:00:00
- KEY(3: DMLTYPE$$) = I
- KEY(4: OLD_NEW$$) = N
- KEY(5: CHANGE_VECTOR$$) = [B@246ae04d
- KEY(6: XID$$) = 1688965824382101
[flysync@oracle40 ~]$
c.在目标端（node2）验证kufl 已装载。
[flysync@node2 ~]$ kufl list -last 1
SEQ# = 58 / FRAG# = 0 (last frag)
- TIME = 2023-11-19 01:10:58.0
- EPOCH# = 57
- EVENTID = ora:1178165:1178299
- SOURCEID = oracle40
- METADATA = [dbms_type=oracle;tz_aware=true;xid=1688965824382101;tz_aware=true;all_tables=[OA11G.EMP, OA11G.MLOG$_EMP, OA11G.DEPT];service=oracle;shard=OA11G]
- TYPE = com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS = []
- SQL(0) =
- ACTION = INSERT
- SCHEMA = oa11g0017
- TABLE = emp0017
- ROW# = 0
- COL(1: empno) = 7935
- COL(2: ename) = KFS
- COL(3: job) = NULL
- COL(4: mgr) = NULL
- COL(5: hiredate) = NULL
- COL(6: sal) = NULL
- COL(7: comm) = NULL
- COL(8: deptno) = NULL
- SQL(1) =
- ACTION = INSERT
- SCHEMA = oa11g0017
- TABLE = mlog$_emp
- ROW# = 0
- COL(1: empno) = 7935
- COL(2: snaptime$$) = 4000-01-01 00:00:00
- COL(3: dmltype$$) = I
- COL(4: old_new$$) = N
- COL(5: change_vector$$) = [B@68837a77
- COL(6: xid$$) = 1688965824382101
- SQL(2) =
- ACTION = INSERT
- SCHEMA = oa11g0017
- TABLE = dept
- ROW# = 0
- COL(1: deptno) = 50
- COL(2: dname) = engineer
- COL(3: loc) = NEW YORK
- SQL(3) =
- ACTION = DELETE
- SCHEMA = oa11g0017
- TABLE = mlog$_emp
- ROW# = 0
- KEY(1: empno) = 7935
- KEY(2: snaptime$$) = 4000-01-01 00:00:00
- KEY(3: dmltype$$) = I
- KEY(4: old_new$$) = N
- KEY(5: change_vector$$) = [B@6be46e8f
- KEY(6: xid$$) = 1688965824382101
[flysync@node2 ~]$
d.在目标端（node2）验证数据已同步
[flysync@node2 ~]$ exit
登出
[root@node2 ~]#
[root@node2 ~]# su - kingbase
上一次登录：日 11月 19 00:56:10 CST 2023pts/0 上
[kingbase@node2 ~]$ ksql -h node2 -Usystem -dtest
用户 system 的口令：
ksql (V8.0)
输入 "help" 来获取帮助信息.
test=# SELECT * FROM oa11g0017.emp0017 WHERE empno=7935;
empno | ename | job | mgr | hiredate | sal | comm | deptno
-------+-------+-----+-----+----------+-----+------+--------
7935 | KFS | | | | | |
(1 行记录)
test=# SELECT * FROM oa11g0017.dept WHERE deptno=50;
deptno | dname | loc
--------+----------+----------
50 | engineer | NEW YORK
(1 行记录)
test=#
(5).增量数据同步(kingbase为主，Oracle 为备)
1.指定主库和备库，ONLINE同步服务，数据开始同步
指定主库和备库，ONLINE同步服务，数据开始同步
[kingbase@node2 ~]$ exit
登出
[root@node2 ~]# su - flysync
上一次登录：日 11月 19 01:13:15 CST 2023pts/0 上
[flysync@node2 ~]$ repswitch -conf switch_conf show
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing show command...
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
NAME VALUE
---- -----
configName : oracle40
sourceId : oracle40
role : master
resourceVendor : oracle
state : ONLINE
masterListenUri : kufl://oracle40:3112/
uptimeSeconds : 6556.822
appliedLastSeqno: 58
NAME VALUE
---- -----
configName : node2
sourceId : node2
role : slave
resourceVendor : kingbase
state : ONLINE
masterListenUri : kufl://node2:3112/
uptimeSeconds : 2191.032
appliedLastSeqno: 58
Finish show command...
[flysync@node2 ~]$ repswitch -conf switch_conf offline
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing offline command...
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
oracle40 offline sucessful
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
node2 offline sucessful
Finish offline command...
[flysync@node2 ~]$ repswitch -conf switch_conf online node2
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
Processing online command...
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
| [ - main] INFO database.AbstractDatabase Loading database driver: com.kingbase8.Driver
Set extractor start specific event ID:kb:30331280
put node node2 as master to online sucessful
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
put node oracle40 as slave to online sucessful
Finish online command...
[flysync@node2 ~]$ repswitch -conf switch_conf show
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
| [ - main] INFO util.ConnectMBean conn start
| [ - main] INFO util.ConnectMBean Connected to replicator process
| [ - main] INFO util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing show command...
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
| [ - main] INFO util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
NAME VALUE
---- -----
configName : oracle40
sourceId : oracle40
role : slave
resourceVendor : oracle
state : ONLINE
masterListenUri : kufl://oracle40:3112/
uptimeSeconds : 6747.664
appliedLastSeqno: 59
NAME VALUE
---- -----
configName : node2
sourceId : node2
role : master
resourceVendor : kingbase
state : ONLINE
masterListenUri : kufl://node2:3112/
uptimeSeconds : 2381.875
appliedLastSeqno: 59
Finish show command...
2.查看同步进程状态
a.Oracle端(oracle40)
[flysync@oracle40 ~]$ fsrepctl -service oracle status
Processing status command...
NAME VALUE
---- -----
appliedLastEventId : kb:30501104:30501936
appliedLastSeqno : 59
appliedLatency : 0.336
autoRecoveryEnabled : false
autoRecoveryTotal : 0
channels : 1
clusterName : oracle
currentEventId : NONE
currentTimeMillis : 1700328051586
dataServerHost : oracle40
extensions :
host : oracle40
latestEpochNumber : 59
masterConnectUri : kufl://node2:3112/
masterListenUri : kufl://oracle40:3112/
maximumStoredSeqNo : 59
minimumStoredSeqNo : 0
offlineRequests : NONE
pendingError : NONE
pendingErrorCode : NONE
pendingErrorEventId : NONE
pendingErrorSeqno : -1
pendingExceptionMessage: NONE
pipelineSource : kufl://node2:3112/
relativeLatency : 167.112
resourcePrecedence : 99
rmiPort : 11000
role : slave
seqnoType : java.lang.Long
serviceName : oracle
serviceType : local
simpleServiceName : oracle
siteName : default
sourceId : oracle40
state : ONLINE
timeInStateSeconds : 166.803
timezone : GMT+08:00
transitioningTo :
uptimeSeconds : 6725.988
useSSLConnection : false
version : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@oracle40 ~]$
b.KES端(node2)
[flysync@node2 ~]$ fsrepctl -service kingbase8 status
Processing status command...
NAME VALUE
---- -----
appliedLastEventId : kb:30501104:30501936
appliedLastSeqno : 59
appliedLatency : 0.069
autoRecoveryEnabled : false
autoRecoveryTotal : 0
channels : 1
clusterName : kingbase8
currentEventId : kb:30501104:30502968
currentTimeMillis : 1700328169274
dataServerHost : node2
extensions :
host : node2
latestEpochNumber : 59
masterConnectUri : kufl://oracle40:3112/
masterListenUri : kufl://node2:3112/
maximumStoredSeqNo : 59
minimumStoredSeqNo : 0
offlineRequests : NONE
pendingError : NONE
pendingErrorCode : NONE
pendingErrorEventId : NONE
pendingErrorSeqno : -1
pendingExceptionMessage: NONE
pipelineSource : UNKNOWN
relativeLatency : 284.801
resourcePrecedence : 99
rmiPort : 11000
role : master
seqnoType : java.lang.Long
serviceName : kingbase8
serviceType : local
simpleServiceName : kingbase8
siteName : default
sourceId : node2
state : ONLINE
timeInStateSeconds : 284.788
timezone : GMT+08:00
transitioningTo :
uptimeSeconds : 2477.872
useSSLConnection : false
version : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@node2 ~]$
3.数据同步验证(验证DML)
a.在源端（node2）生成数据
[flysync@node2 ~]$ exit
登出
[root@node2 ~]# su - kingbase
上一次登录：日 11月 19 01:14:28 CST 2023pts/0 上
[kingbase@node2 ~]$ ksql -h node2 -Usystem -dtest
用户 system 的口令：
ksql (V8.0)
输入 "help" 来获取帮助信息.
test=# \d oa11g0017.emp0017;
数据表 "oa11g0017.emp0017"
栏位 | 类型 | 校对规则 | 可空的 | 预设
----------+-----------------------------+----------+----------+------
empno | integer | | not null |
ename | character varying(10 char) | | |
job | character varying(9 char) | | |
mgr | integer | | |
hiredate | timestamp without time zone | | |
sal | numeric(7,2) | | |
comm | numeric(7,2) | | |
deptno | smallint | | |
索引：
"emp0017_pkey" PRIMARY KEY, btree (empno)
test=# \d oa11g0017.dept;
数据表 "oa11g0017.dept"
栏位 | 类型 | 校对规则 | 可空的 | 预设
--------+----------------------------+----------+----------+------
deptno | smallint | | not null |
dname | character varying(14 char) | | |
loc | character varying(13 char) | | |
索引：
"dept_pkey" PRIMARY KEY, btree (deptno)
test=# INSERT INTO oa11g0017.emp0017(EMPNO, ENAME) VALUES(7936,'KDT')
test-# ;
INSERT 0 1
test=# INSERT INTO oa11g0017.dept VALUES(51,'sales','Santiago');
INSERT 0 1
test=#
b.查看源端（node2）kufl 是否解析
[kingbase@node2 ~]$ exit
登出
[root@node2 ~]# su - flysync
上一次登录：日 11月 19 01:18:01 CST 2023pts/0 上
[flysync@node2 ~]$ kufl list -last 2
SEQ# = 60 / FRAG# = 0 (last frag)
- TIME = 2023-11-19 01:24:50.134
- EPOCH# = 59
- EVENTID = kb:30503232:30504912
- SOURCEID = node2
- METADATA = [tz_aware=true;dbms_type=kingbase;client_id;all_tables=[oa11g0017.emp0017];service=kingbase8;shard=oa11g0017]
- TYPE = com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS = []
- SQL(0) =
- ACTION = INSERT
- SCHEMA = oa11g0017
- TABLE = emp0017
- ROW# = 0
- COL(1: empno) = 7936
- COL(2: ename) = KDT
- COL(3: job) = NULL
- COL(4: mgr) = NULL
- COL(5: hiredate) = NULL
- COL(6: sal) = NULL
- COL(7: comm) = NULL
- COL(8: deptno) = NULL
SEQ# = 61 / FRAG# = 0 (last frag)
- TIME = 2023-11-19 01:25:04.2
- EPOCH# = 59
- EVENTID = kb:30506088:30506824
- SOURCEID = node2
- METADATA = [tz_aware=true;dbms_type=kingbase;client_id;all_tables=[oa11g0017.dept];service=kingbase8;shard=oa11g0017]
- TYPE = com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS = []
- SQL(0) =
- ACTION = INSERT
- SCHEMA = oa11g0017
- TABLE = dept
- ROW# = 0
- COL(1: deptno) = 51
- COL(2: dname) = sales
- COL(3: loc) = Santiago
[flysync@node2 ~]$
c.在目标端（oracle40）验证kufl 已装载。
[flysync@oracle40 ~]$ kufl list -last 2
SEQ# = 59 / FRAG# = 0 (last frag)
- TIME = 2023-11-19 01:18:04.473
- EPOCH# = 59
- EVENTID = kb:30501104:30501936
- SOURCEID = node2
- METADATA = [tz_aware=true;dbms_type=kingbase;client_id;all_tables=[flysync_kingbase8.heartbeat];is_metadata=true;service=kingbase8;shard=#UNKNOWN;source_service_name=flysync_kingbase8;heartbeat=MASTER_ONLINE]
- TYPE = com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS = []
- SQL(0) =
- ACTION = UPDATE
- SCHEMA = FLYSYNC
- TABLE = heartbeat
- ROW# = 0
- COL(1: id) = 1
- COL(2: seqno) = 53
- COL(3: eventid) = ora:1177078:1177095
- COL(4: source_tstamp) = 2023-11-19 01:18:04.471
- COL(5: target_tstamp) = 2023-11-19 01:00:08.727
- COL(6: lag_millis) = 780721
- COL(7: salt) = 1
- COL(8: name) = MASTER_ONLINE
- KEY(1: id) = 1
SEQ# = 60 / FRAG# = 0 (last frag)
- TIME = 2023-11-19 01:24:50.134
- EPOCH# = 59
- EVENTID = kb:30503232:30504912
- SOURCEID = node2
- METADATA = [tz_aware=true;dbms_type=kingbase;client_id;all_tables=[oa11g0017.emp0017];service=kingbase8;shard=oa11g0017]
- TYPE = com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS = []
- SQL(0) =
- ACTION = INSERT
- SCHEMA = OA11G
- TABLE = EMP0017
- ROW# = 0
- COL(1: EMPNO) = 7936
- COL(2: ENAME) = KDT
- COL(3: JOB) = NULL
- COL(4: MGR) = NULL
- COL(5: HIREDATE) = NULL
- COL(6: SAL) = NULL
- COL(7: COMM) = NULL
- COL(8: DEPTNO) = NULL
[flysync@oracle40 ~]$
d.在目标端（oracle40）验证数据已同步
[root@oracle40 ~]# su - oracle
Last login: Sun Nov 19 15:19:13 CST 2023 on pts/0
[oracle@oracle40 ~]$ sqlplus OA11G/oa11g
SQL*Plus: Release 11.2.0.4.0 Production on Sun Nov 19 15:25:30 2023
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL>
SQL> SELECT * FROM oa11g.emp WHERE empno=7936;
EMPNO ENAME JOB MGR HIREDATE SAL COMM
---------- ---------- --------- ---------- --------- ---------- ----------
DEPTNO
----------
7936 KDT
SQL> SELECT * FROM oa11g.dept WHERE deptno=51;
DEPTNO DNAME LOC
---------- -------------- -------------
51 sales Santiago
SQL>
