【用户投稿】Apache SeaTunnel 2.3.3+Web 1.0.0版本安装部署

news2024/12/26 22:45:30

项目概要

Apache SeaTunnel 是一个分布式、高性能、易扩展的数据集成平台，用于实时和离线数据处理,支持多种数据源之间的数据迁移和转换。
其中，Apache-seatunnel-web-1.0.0-bin.tar.gz和apache-seatunnel-2.3.3-bin.tar.gz代表了 Apache SeaTunnel Web 界面的安装包和 Apache SeaTunnel 的核心引擎的安装包。

  • **apache-seatunnel-web-1.0.0-bin.tar.gz：**这是 Apache SeaTunnel Web 的安装包，提供了用户友好的 Web 界面，使得用户可以更方便地管理和监控 SeaTunnel 任务，包括创建、编辑和运行作业等操作。通过这个界面，用户可以无需直接接触命令行就能进行配置和管理。
  • **apache-seatunnel-2.3.3-bin.tar.gz：**这是 Apache SeaTunnel 核心引擎的二进制发行版，包含了执行数据抽取、转换和加载的核心代码库。它是实际执行数据处理任务的部分，可以与各种数据源和目标（如数据库、文件系统、消息队列等）进行交互。
    在部署 Apache SeaTunnel 时，通常需要先安装并配置好核心引擎，然后根据需要选择是否使用 Web 界面进行管理。如果你只需要在命令行下运行和管理任务，那么只需要安装核心引擎即可。如果你想拥有一个图形化的任务管理界面，那么就需要同时安装 Apache SeaTunnel Web。

file

安装包及源码包下载地址：https://seatunnel.apache.org/download/

file

在部署 Apache SeaTunnel 时，通常需要先安装并配置好核心引擎，然后根据需要选择是否使用 Web 界面进行管理。如果你只需要在命令行下运行和管理任务，那么只需要安装核心引擎即可。如果你想拥有一个图形化的任务管理界面，那么就需要同时安装 Apache SeaTunnel Web。

核心引擎部署

上传解压安装包： tar xf apache-seatunnel-2.3.3-bin.tar.gz

file

自动安装

自动安装插件（全量安装，速度比较慢，一直到下班，一个半小时都没结束，第二天早上来显示下载结束）

[opensource@bigdata02 bin]$ pwd
/home/opensource/app/apache-seatunnel-2.3.3/bin
[opensource@bigdata02 bin]$ ls
install-plugin.sh     seatunnel.sh                              start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh   stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$ 
[opensource@bigdata02 bin]$ ./install-plugin.sh

file

验证服务

执行系统自带的任务进行验证： sh seatunnel.sh --config ../config/v2.batch.config.template -m local

file

[opensource@bigdata02 bin]$ ls
install-plugin.sh     seatunnel.sh                              start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh   stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$ 
[opensource@bigdata02 bin]$ nohup sh seatunnel-cluster.sh 2>&1 &    
[1] 93523
[opensource@bigdata02 bin]$ nohup: 忽略输入并把输出追加到"nohup.out"

[opensource@bigdata02 bin]$ ls
install-plugin.sh  seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh  stop-seatunnel-cluster.sh
nohup.out          seatunnel.sh          start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
[opensource@bigdata02 bin]$ tail -f nohup.out 
十二月 08, 2023 9:30:58 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
^C
[opensource@bigdata02 bin]$ tail -f ../logs/seatunnel-engine-server.log 
        Member [localhost]:5801 - 9db58520-2335-441c-8999-ad78123f6509 this
]

2023-12-08 09:30:59,930 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel] [5.1] [localhost]:5801 is STARTED
2023-12-08 09:31:01,865 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] This node become a new active master node, begin init coordinator service
2023-12-08 09:31:02,061 INFO  com.hazelcast.internal.partition.impl.PartitionStateManager - [localhost]:5801 [seatunnel] [5.1] Initializing cluster partition table arrangement...
2023-12-08 09:31:04,863 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - Init ResourceManager
2023-12-08 09:31:04,863 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker... 
2023-12-08 09:31:04,864 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker live nodes: [[localhost]:5801]
2023-12-08 09:31:04,865 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, assignedSlots=[], unassignedSlots=[])}
2023-12-08 09:31:59,863 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] 
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   0
corePoolSize              :                   0
maximumPoolSize           :          2147483647
poolSize                  :                   0
completedTaskCount        :                   0
taskCount                 :                   0
***********************************************

2023-12-08 09:31:59,867 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] 
***********************************************
                Job info detail
***********************************************
createdJobCount           :                   0
scheduledJobCount         :                   0
runningJobCount           :                   0
failingJobCount           :                   0
failedJobCount            :                   0
cancellingJobCount        :                   0
canceledJobCount          :                   0
finishedJobCount          :                   0
restartingJobCount        :                   0
suspendedJobCount         :                   0
reconcilingJobCount       :                   0
***********************************************

启动服务

启动服务并且后台运行，用于web端连接：nohup sh seatunnel-cluster.sh 2>&1 &

file

[opensource@bigdata02 bin]$ ls
install-plugin.sh     seatunnel.sh                              start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh   stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$ 
[opensource@bigdata02 bin]$ nohup sh seatunnel-cluster.sh 2>&1 &    
[1] 93523
[opensource@bigdata02 bin]$ nohup: 忽略输入并把输出追加到"nohup.out"

[opensource@bigdata02 bin]$ ls
install-plugin.sh  seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh  stop-seatunnel-cluster.sh
nohup.out          seatunnel.sh          start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
[opensource@bigdata02 bin]$ tail -f nohup.out 
十二月 08, 2023 9:30:58 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
^C
[opensource@bigdata02 bin]$ tail -f ../logs/seatunnel-engine-server.log 
        Member [localhost]:5801 - 9db58520-2335-441c-8999-ad78123f6509 this
]

2023-12-08 09:30:59,930 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel] [5.1] [localhost]:5801 is STARTED
2023-12-08 09:31:01,865 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] This node become a new active master node, begin init coordinator service
2023-12-08 09:31:02,061 INFO  com.hazelcast.internal.partition.impl.PartitionStateManager - [localhost]:5801 [seatunnel] [5.1] Initializing cluster partition table arrangement...
2023-12-08 09:31:04,863 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - Init ResourceManager
2023-12-08 09:31:04,863 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker... 
2023-12-08 09:31:04,864 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker live nodes: [[localhost]:5801]
2023-12-08 09:31:04,865 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, assignedSlots=[], unassignedSlots=[])}
2023-12-08 09:31:59,863 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] 
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   0
corePoolSize              :                   0
maximumPoolSize           :          2147483647
poolSize                  :                   0
completedTaskCount        :                   0
taskCount                 :                   0
***********************************************

2023-12-08 09:31:59,867 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] 
***********************************************
                Job info detail
***********************************************
createdJobCount           :                   0
scheduledJobCount         :                   0
runningJobCount           :                   0
failingJobCount           :                   0
failedJobCount            :                   0
cancellingJobCount        :                   0
canceledJobCount          :                   0
finishedJobCount          :                   0
restartingJobCount        :                   0
suspendedJobCount         :                   0
reconcilingJobCount       :                   0
***********************************************

手动安装

不想全部安装，修改配置：vi config/plugin_config

file

无法在线安装，只能手动安装，maven库下载地址：https://repo.maven.apache.org/maven2/org/apache/seatunnel/

file

对于自动下载下来的jar包，已经全部打包上传到CSDN,大家可以免费下载，直接放到connectors/seatunnel

file

下载地址：https://download.csdn.net/download/qq_36434219/88609069

file

CSDN资源大小不允许超过1000M,这里采用.tar.xz格式（解压会有些慢，小十分钟吧）

三种压缩命令：

tar -cf seatunnel_connectors_2.3.3.tar seatunnel
tar -czf seatunnel_connectors_2.3.3.tar.gz seatunnel
tar -cJf seatunnel_connectors_2.3.3.tar.xz seatunnel

对应解压命令：

tar -xf seatunnel_connectors_2.3.3.tar 
tar -xzf seatunnel_connectors_2.3.3.tar.gz 
tar -xJf seatunnel_connectors_2.3.3.tar.xz

file

Web界面部署

上传解压web部署包：tar xf apache-seatunnel-web-1.0.0-bin.tar.gz

初始化数据库

初始化SQL脚本 script/seatunnel_server_mysql.sql

file

file

SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------
-- Table structure for role
-- ----------------------------
DROP TABLE IF EXISTS `role`;
CREATE TABLE `role`  (
  `id` int(20) NOT NULL AUTO_INCREMENT,
  `type` int(2) NOT NULL,
  `role_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Records of role
-- ----------------------------
INSERT INTO `role`(`type`,`role_name`,`description`) values (0, 'ADMIN_ROLE', 'Admin User');
INSERT INTO `role`(`type`,`role_name`,`description`) values (1, 'NORMAL_ROLE', 'Normal User');

-- ----------------------------
-- Table structure for role_user_relation
-- ----------------------------
DROP TABLE IF EXISTS `role_user_relation`;
CREATE TABLE `role_user_relation`  (
  `id` int(20) NOT NULL AUTO_INCREMENT,
  `role_id` int(20) NOT NULL,
  `user_id` int(20) NOT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_datasource
-- ----------------------------
DROP TABLE IF EXISTS `t_st_datasource`;
CREATE TABLE `t_st_datasource`  (
  `id` bigint(20) NOT NULL,
  `datasource_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `plugin_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `plugin_version` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT '1.0.0',
  `datasource_config` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `description` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `t_st_datasource_datasource_name_uindex`(`datasource_name`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_definition
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_definition`;
CREATE TABLE `t_st_job_definition`  (
  `id` bigint(20) NOT NULL,
  `name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `job_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `name`(`name`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_instance
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_instance`;
CREATE TABLE `t_st_job_instance`  (
  `id` bigint(20) NOT NULL,
  `job_define_id` bigint(20) NOT NULL,
  `job_status` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `job_config` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `engine_name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `engine_version` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `job_engine_id` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(20) NOT NULL,
  `update_user_id` int(20) NULL DEFAULT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  `end_time` timestamp(3) NULL DEFAULT NULL,
  `job_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_instance_history
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_instance_history`;
CREATE TABLE `t_st_job_instance_history`  (
  `id` bigint(20) NOT NULL,
  `dag` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_line
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_line`;
CREATE TABLE `t_st_job_line`  (
  `id` bigint(20) NOT NULL,
  `version_id` bigint(20) NOT NULL,
  `input_plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `target_plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  INDEX `job_line_version_index`(`version_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_metrics
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_metrics`;
CREATE TABLE `t_st_job_metrics`  (
  `id` bigint(20) NOT NULL,
  `job_instance_id` bigint(20) NOT NULL,
  `pipeline_id` int(20) NOT NULL,
  `read_row_count` bigint(20) NOT NULL,
  `write_row_count` bigint(20) NOT NULL,
  `source_table_names` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `sink_table_names` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `read_qps` bigint(20) NULL DEFAULT NULL,
  `write_qps` bigint(20) NULL DEFAULT NULL,
  `record_delay` bigint(20) NULL DEFAULT NULL,
  `status` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(20) NOT NULL,
  `update_user_id` int(20) NULL DEFAULT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_task
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_task`;
CREATE TABLE `t_st_job_task`  (
  `id` bigint(20) NOT NULL,
  `version_id` bigint(20) NOT NULL,
  `plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `config` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
  `transform_options` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `output_schema` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
  `connector_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `datasource_id` bigint(20) NULL DEFAULT NULL,
  `datasource_option` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `select_table_fields` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `scene_mode` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  INDEX `job_task_plugin_id_index`(`plugin_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_version
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_version`;
CREATE TABLE `t_st_job_version`  (
  `id` bigint(20) NOT NULL,
  `job_id` bigint(20) NOT NULL,
  `name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `job_mode` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `env` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
  `engine_name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `engine_version` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_virtual_table
-- ----------------------------
DROP TABLE IF EXISTS `t_st_virtual_table`;
CREATE TABLE `t_st_virtual_table`  (
  `id` bigint(20) NOT NULL,
  `datasource_id` bigint(20) NOT NULL,
  `virtual_database_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `virtual_table_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `table_fields` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `virtual_table_config` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `description` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for user
-- ----------------------------
DROP TABLE IF EXISTS `user`;
CREATE TABLE `user`  (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `username` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `password` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `status` tinyint(4) NOT NULL,
  `type` tinyint(4) NOT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 2 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for user_login_log
-- ----------------------------
DROP TABLE IF EXISTS `user_login_log`;
CREATE TABLE `user_login_log`  (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `user_id` int(11) NOT NULL,
  `token` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `token_status` tinyint(1) NOT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 106 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Records of user_login_log
-- ----------------------------

INSERT INTO `user`(`username`,`password`,`status`,`type`) values ('admin', '7f97da8846fed829bb8d1fd9f8030f3b', 0, 0);

SET FOREIGN_KEY_CHECKS = 1;

配置文件修改

配置数据库信息： vi conf/application.yml

file

复制引擎服务中配置文件到Web配置目录下面

[opensource@bigdata02 app]$ cp apache-seatunnel-2.3.3/config/hazelcast-client.yaml apache-seatunnel-web-1.0.0-bin/conf/
[opensource@bigdata02 app]$ cp apache-seatunnel-2.3.3/connectors/plugin-mapping.properties apache-seatunnel-web-1.0.0-bin/conf/

file

配置下载数据源jar包

选择的Mysql作为元数据库，对应的驱动包mysql-connector-java-8.0.25.jar放到libs下

file

下载其它数据源资源包,首先下载源码，然后执行下载脚本download_datasource.sh(本地安装过git)

源码地址:https://github.com/apache/seatunnel-web

file

修改download_datasource.sh相关配置

file

file

执行download_datasource.sh

file

开始下载jar包

file

下载完成

file

上传到libs目录

file

启动服务

sh bin/seatunnel-backend-daemon.sh start

file

Web页面一览

登录

登录地址：http://xxxx:8801/ui/#/login ;
用户密码 admin/admin
file

file

数据源
file
file

任务
file

尝试创建整库同步
file

用户管理
file

虚拟表
file

其他

问题记录

服务启动失败：Application run failed org.yaml.snakeyaml.scanner.ScannerException: while scanning an anchor

14:29:08.032 [main] ERROR org.springframework.boot.SpringApplication - Application run failed
org.yaml.snakeyaml.scanner.ScannerException: while scanning an anchor
 in 'reader', line 30, column 15:
        password: &R7#J*vDVWEuT48m
                  ^
unexpected character found *(42)
 in 'reader', line 30, column 20:
        password: &R7#J*vDVWEuT48m
                       ^

        at org.yaml.snakeyaml.scanner.ScannerImpl.scanAnchor(ScannerImpl.java:1507)
        at org.yaml.snakeyaml.scanner.ScannerImpl.fetchAnchor(ScannerImpl.java:958)
        at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:389)
        at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:248)
        at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingValue.produce(ParserImpl.java:665)
        at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:165)
        at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:59)
        at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:45)
        at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:140)
        at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:119)
        at org.yaml.snakeyaml.composer.Composer.composeScalarN

解决办法： 密码&R7#J*vDVWEuT48m增加双引号

file

file

原文链接：https://blog.csdn.net/qq_36434219/article/details/134857560

本文由 白鲸开源科技 提供发布支持！

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.coloradmin.cn/o/1619483.html

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈，一经查实，立即删除！

相关文章

FIR补偿滤波器——matlab的FDA实现

FIR补偿滤波器——matlab的FDA实现

输入采样频率：192KHz 抽取倍数：2 通带截至频率：20KHz 通带衰减：0.1dB 阻带衰减：120dB 在更多选项那里，设置c为0.5，代表抽取倍数为1/c，p设置为4，代表级联阶数。FIR补偿…
阅读更多...
新生儿听力筛查怎么做？宝爸宝妈必看

新生儿听力筛查怎么做？宝爸宝妈必看

健康讲堂 根据全国第二次残疾人抽样调查结果显示，目前我国现有听力残疾人2780万，其中17岁以下单纯听力残疾儿童22.15万。新生儿听力障碍发生率约为1～3‰，如果没有得到早期有效的治疗，势必影响其情感、智力的正常发育&a…
阅读更多...
CCFC3007 串口中断问题和W1C记录

CCFC3007 串口中断问题和W1C记录

最近在看国芯的一个串口中断问题。 串口发送完成中断和接收中断是这样写的。 寄存器描述是这样的： 发送完成和接收完成中断，写1清除。 乍一看，上面的中断处理函数写的应该没问题，但其实这样写是有隐患的。 先来看段程序&#x…
阅读更多...
大模型应用RAG系列（1）初识RAG

大模型应用RAG系列（1）初识RAG

题外话 之前我们在讲大模型的应用方向和架构时，有提到RAG、Agent、Fine-Tune。在作者写大模型专题的文章时，也是边学习，边梳理，边总结。在这个过程中，大模型在各个方向都不断地快速发展，对应的paper、理论…
阅读更多...
第55篇：创建Nios II工程之Hello_World＜一＞

第55篇：创建Nios II工程之Hello_World＜一＞

Q：本期我们开始介绍创建Platform Designer系统，并设计基于Nios II Professor的Hello_world工程。 A：设计流程和实验原理：需要用到的IP组件有Clock Source、Nios II Professor、On-Chip Memory、JTAG UART和System ID外设。Nios I…
阅读更多...
YesPMP众包平台最新项目

YesPMP众包平台最新项目

YesPMP一站式互联网众包平台，最新外包项目，有感兴趣的用户可进入平台参与竞标。 （竞标后由项目方直接与服务商联系，双方直接对接） 1.查看项目：个人技术-YesPMP平台 2.查看项目&#xff1…
阅读更多...
SpringBoot 启动控制台 --banner.txt实现打印炫酷控制台图案

SpringBoot 启动控制台 --banner.txt实现打印炫酷控制台图案

文章目录 目录 文章目录 安装流程 小结 概要安装流程技术细节小结 概要 分析源代码，banner.txt实现打印控制台 控制台图案生成网址：Ascii艺术字实现个性化Spring Boot启动banner图案，轻松修改更换banner.txt文件内容，收集了丰富…
阅读更多...
数据结构之顺序表（java版）

数据结构之顺序表（java版）

目录 一.线性表 1.1线性表的概念 二.顺序表 2.1顺序表的概念 2.2顺序表的实现 1.顺序表的接口 1.2顺序表的功能实现 1.顺序表初始化 2.新增元素功能： 3.清空顺序表是否为空&&获取顺序表长度&&打印顺序表： 4.判断是否包含某个…
阅读更多...
vue2和vue3浏览器兼容性对比

vue2和vue3浏览器兼容性对比

一、Vue2.0不支持IE8, 因为Vue使用了IE8无法模拟的ECMAScript 5 特性,但它支持所有兼容ECMAScript 5 的浏览器。 二、Vue3.0 不支持 IE11 及以下版本。兼容ECMAScript 5的浏览器
阅读更多...
MappedStatement解析流程

MappedStatement解析流程

前言 之前写了一篇博文，介绍了mybatis的解析过程，其中mapper标签只演示了如何使用，这篇博文我们来探究mapper标签解析流程 源码解析 核心方法入口 引入mapper方式 使用相对于类路径的资源引用使用完全限定资源定位符（URL&…
阅读更多...
简易频率计的设计与验证

简易频率计的设计与验证

1.原理 频率测量法缺点：时间T确定，但不能保证被测时钟信号在测量时间T内的时钟周期是整数倍。N出现误差那么 结果也会出现误差，而且被测信号的频率越小，T不变，N的个数就会变小，而误差是不变的。N越小误差对…
阅读更多...
mac如何通过Teminal查找本机ip地址

mac如何通过Teminal查找本机ip地址

方法1 ifconfig ifconfig终端中使用 ifconfig 可以查看本地 ip 但是看到的信息太多了，我们只是想要一个本机ip而已 方法2 ifconfig en0 ifconfig en0相比方法1我们算是直接获取到了本机ip，但是第一眼看到还是需要反应一段时间 方法3 精准获取IP ifcon…
阅读更多...
一个简单的java递归下降语法分析器例子

一个简单的java递归下降语法分析器例子

import parser.Parser; import parser.RecursiveDescentParser;import java.util.ArrayList; import java.util.Arrays; import java.util.List;public class Main {public static void main(String[] args) {// 关键词List<String> keyList new ArrayList<>(Arra…
阅读更多...
SSH Key生成

SSH Key生成

天行健&#xff0c;君子以自强不息&#xff1b;地势坤&#xff0c;君子以厚德载物。 每个人都有惰性&#xff0c;但不断学习是好好生活的根本&#xff0c;共勉&#xff01; 文章均为学习整理笔记&#xff0c;分享记录为主&#xff0c;如有错误请指正&#xff0c;共同学习进步。…
阅读更多...
G2024-04-24 开源项目日报 Top10

G2024-04-24 开源项目日报 Top10

根据Github Trendings的统计，今日(2024-04-24统计)共有10个项目上榜。根据开发语言中项目的数量，汇总情况如下： 开发语言项目数量Python项目5TypeScript项目3PowerShell项目1C++项目1非开发语言项目1LLaMA模型的C/C++移植 创建周期：306 天开发语言：C, C++协议类型：MIT Li…
阅读更多...
详解数据结构：队列（含栈与队列扩展）

详解数据结构：队列（含栈与队列扩展）

一、顺序队列 有一种线性序列&#xff0c;特点是先进先出&#xff0c;这种存储结构称为队列。队列也是一种线性表&#xff0c;只不过它是操作受限的线性表&#xff0c;只能再两端操作&#xff1a;一端进、一端出。进的一端称为队尾&#xff0c;出的一端称为队头。队列可以用顺…
阅读更多...
西电超算使用方法-简易版

西电超算使用方法-简易版

一、引言 西电超算不错&#xff0c;我很喜欢。本文仅供自己学习使用。 二、环境搭建 搭建环境需要有一些依赖库&#xff0c;但是其实西电超算说明手册并没有写的非常清楚。因此&#xff0c;这次实战演示一下&#xff0c;写一个运行sh文件脚本并提交作业。 1、选择GPU还是CP…
阅读更多...
[计算机效率] 网站推荐：文字转语音

[计算机效率] 网站推荐：文字转语音

4.7 文字转语音 文字转语音的作用主要体现在以下几个方面&#xff1a; 便捷性&#xff1a;文字转语音功能为视觉障碍者提供了极大的便利。通过语音输出&#xff0c;他们可以更轻松地获取和理解信息&#xff0c;从而在日常生活中更加独立自主。 节省时间&#xff1a;对于那些需…
阅读更多...
沐风老师3dMax万有引力插件ToGround使用方法详解

沐风老师3dMax万有引力插件ToGround使用方法详解

3dMax万有引力插件ToGround使用教程 3dMax万有引力插件ToGround&#xff0c;用于在复杂地形&#xff08;曲面&#xff09;上将对象放置在适当高度的实用工具。例如&#xff1a;将大量的人、植物和汽车快速放置在一个街道、公园和小跑道高度不同的区域尤其有用。 【适用版本】 …
阅读更多...
工信部政策要求试点城市20%资金奖励中小企业用SaaS上云转型数字化

工信部政策要求试点城市20%资金奖励中小企业用SaaS上云转型数字化

随着数字经济的不断发展&#xff0c;中小企业也迎来了前所未有的机遇和挑战。为了持续推动中小企业数字化转型&#xff0c;工信部出台了一项新政策&#xff0c;主要通过资金奖励的方式&#xff0c;鼓励中小企业采纳软件即服务&#xff08;SaaS&#xff09;模式&#xff0c;实现…
阅读更多...
推荐文章
最新文章

Copyright @ 2022~2023