【flink】Chunk splitting has encountered exception

news2025/1/18 17:12:46

执行任务报错: Chunk splitting has encountered exception

错误信息截图:

 

完整的错误信息:

16:30:43,911 ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator [SourceCoordinator-Source: CDC Sourceorg.jobslink.flink.UserCDCJob]  - Uncaught exception in the SplitEnumerator for Source Source: CDC Sourceorg.flink.UserCDCJob while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover.
org.apache.flink.util.FlinkRuntimeException: Chunk splitting has encountered exception
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.checkSplitterErrors(MySqlSnapshotSplitAssigner.java:522)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:278)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:298)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:123)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:201)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:117)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$2(SourceCoordinator.java:230)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$9(SourceCoordinator.java:406)
	at org.apache.flink.util.ThrowableCatchingRunnable.run(ThrowableCatchingRunnable.java:40)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Error when splitting chunks for jobslink_user.jobslink_account
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:246)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitChunksForRemainingTables(MySqlSnapshotSplitAssigner.java:505)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	... 3 more
Caused by: java.lang.RuntimeException: Fail to analyze table in chunk splitter.
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:154)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.splitChunks(MySqlChunkSplitter.java:117)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:244)
	... 7 more
Caused by: org.apache.flink.table.api.ValidationException: Incremental snapshot for tables requires primary key, but table user.account doesn't have primary key.
	at com.ververica.cdc.connectors.mysql.source.utils.ChunkUtils.getChunkKeyColumn(ChunkUtils.java:55)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:148)
	... 9 more
16:30:43,929 INFO  org.apache.flink.runtime.jobmaster.JobMaster                 [flink-akka.actor.default-dispatcher-9]  - Trying to recover from a global failure.
org.apache.flink.util.FlinkException: Global failure triggered by OperatorCoordinator for 'Source: CDC Sourceorg.jobslink.flink.UserCDCJob -> Filter -> Flat Map -> Filter -> (Sink: Print to Std. Out, Sink: sink jobslink_account)' (operator cbc357ccb763df2852fee8c4fc7d55f2).
	at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder$LazyInitializedCoordinatorContext.failJob(OperatorCoordinatorHolder.java:556)
	at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$QuiesceableContext.failJob(RecreateOnResetOperatorCoordinator.java:231)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinatorContext.failJob(SourceCoordinatorContext.java:316)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$9(SourceCoordinator.java:420)
	at org.apache.flink.util.ThrowableCatchingRunnable.run(ThrowableCatchingRunnable.java:40)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.util.FlinkRuntimeException: Chunk splitting has encountered exception
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.checkSplitterErrors(MySqlSnapshotSplitAssigner.java:522)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:278)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:298)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:123)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:201)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:117)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$2(SourceCoordinator.java:230)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$9(SourceCoordinator.java:406)
	... 9 more
Caused by: java.lang.IllegalStateException: Error when splitting chunks for jobslink_user.jobslink_account
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:246)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitChunksForRemainingTables(MySqlSnapshotSplitAssigner.java:505)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	... 3 more
Caused by: java.lang.RuntimeException: Fail to analyze table in chunk splitter.
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:154)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.splitChunks(MySqlChunkSplitter.java:117)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:244)
	... 7 more
Caused by: org.apache.flink.table.api.ValidationException: Incremental snapshot for tables requires primary key, but table user.account doesn't have primary key.
	at com.ververica.cdc.connectors.mysql.source.utils.ChunkUtils.getChunkKeyColumn(ChunkUtils.java:55)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:148)
	... 9 more
16:30:43,933 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [flink-akka.actor.default-dispatcher-9]  - Job org.flink.UserCDCJob (35780b6f0490a368fc256b228e99f7cb) switched from state RUNNING to RESTARTING.
16:30:43,936 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [flink-akka.actor.default-dispatcher-9]  - Source: CDC Sourceorg.flink.UserCDCJob -> Filter -> Flat Map -> Filter -> (Sink: Print to Std. Out, Sink: sink account) (1/1) (45028a2ced14e3bbe2d7cf3349defba0) switched from RUNNING to CANCELING.

 Incremental snapshot for tables requires primary key, but table user.account doesn't have primary key.

表的增量快照需要主键,但表user.account没有主键。

user库account表增加主键

添加上主键以后就可以正常执行了!!! 

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.coloradmin.cn/o/843336.html

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈,一经查实,立即删除!

相关文章

python爬虫3:requests库-案例1

python爬虫3:requests库-案例1 前言 ​ python实现网络爬虫非常简单,只需要掌握一定的基础知识和一定的库使用技巧即可。本系列目标旨在梳理相关知识点,方便以后复习。 申明 ​ 本系列所涉及的代码仅用于个人研究与讨论,并不会对网…

分享由于找不到vcruntime140_1.dll,无法继续执行代码的几种解决方法

vcruntime140_1.dll是Microsoft Visual C Redistributable for Visual Studio 2015中的一个动态链接库文件。它是用于支持在Windows操作系统上运行使用Visual C编写的应用程序或游戏所必需的文件之一。当出现vcruntime140_1.dll丢失的错误时,通常是由于缺少或损坏了…

Python系统学习1-2-计算机基础,python入门技巧

目录 一、硬件 二、软件:程序文档 三、基础知识 四、python执行过程 五、Pycharm使用技巧 一、硬件 计算机五大部件:运算器,存储器,控制器、输入设备,输出设备。 运算器和控制器 集成在CPU中。 存储&#xff1a…

【笔记】移动光猫改桥接

1. 登录后台 移动光猫的超管和密码(百度的) 账号:CMCCAdmin 密码:aDm8H%MdA 浏览器访问 192.168.1.1 并登录 2. 选择连接 点击“网络”,在“连接名称”下拉框选择 INTENET_R_VID 字样的连接,并截图备…

java.lang.IllegalArgumentException: Invalid character found in methodname

postman请求异常:java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens

38K红外通信原理

红外通信原理 接收 以VS1838B红外接收管为例,正面对着你,引脚从左到右依次为OUT,GND,VCC ,电路图为 工作电压2.7~5.5V,红外接收头只接受38K信号(误差范围内)&#xff0…

K8S系列文章之 离线安装自动化工具Ansible

参考 文档 离线安装 Ansible - DevOps - dbaselife 一、Ansible简介 Ansible是一款开源的IT配置管理工具,常被IT界的小伙伴们用于自动化的场景,多用在服务部署、配置管理方面。配置文件采用最常见的yaml格式,学习起来也是比较容易&#xff…

Vue3 列表渲染简单应用

去官网学习→列表渲染 | Vue.js 运行示例&#xff1a; 代码&#xff1a;HelloWorld.vue <template><div class"hello"><h1>Vue 列表渲染</h1><p v-for"item in dataList">{{item}}</p><p v-for"(item,index)…

C++ | C++11新特性(上)

目录 前言 一、列表初始化 二、声明 1、auto 2、decltype 3、nullptr 三、STL容器的变化 四、右值引用与移动语义 1、左值与左值引用 2、右值与右值引用 3、右值引用与左值引用的比较 4、右值引用的场景及意义 &#xff08;1&#xff09;做参数 &#xff08;2&a…

带你了解Python的重要性,就算做测试也要会Python(文末送书)

名字&#xff1a;阿玥的小东东 学习&#xff1a;Python、C/C 主页链接&#xff1a;阿玥的小东东的博客_CSDN博客-python&&c高级知识,过年必备,C/C知识讲解领域博主 目录 人生苦短&#xff0c;我用 Python 在测试工作中用好 Python 进阶&#xff1a;用Python构建测试…

vue2 todoapp案例(静态)

1.创建三个子组件(TodoHeader、TodoMain、TodoFooter)和两个(index.css、base.css)样式&#xff1b; TodoHeader页面 <template><header class"header"><h1>todos</h1><input id"toggle-all" class"toggle-all" typ…

分布式应用:Zabbix监控平台

目录 一、理论 1.Zabbix 2.部署 zabbix 服务端&#xff08;端口号10051&#xff09; 3.部署zabbix 客户端&#xff08;端口号10050&#xff09; 4.将客户端加入服务端的监控主机中 二、实验 1.部署 zabbix 服务端&#xff08;端口号10051&#xff09; 2.部署zabbix 客户…

Postman如何做接口测试

目录 Postman如何做接口测试1&#xff1a;如何导入 swagger 接口文档 Postman如何做接口测试2&#xff1a;如何切换测试环境 Postman如何做接口测试3&#xff1a;什么&#xff1f;postman 还可以做压力测试&#xff1f; Postman如何做接口测试4&#xff1a;如何自动添加请求…

HCIP 三层交换机

一、实现VLAN间通信 在传统的交换机组网中&#xff0c;默认所有网络都处于同一个广播域&#xff0c;带来了许多问题&#xff0c;VLAN技术的提出&#xff0c;满足了二层组网隔离广播域需求&#xff0c;使得属于不同的VLAN间网络无法通信&#xff0c;但不同VLAN之间又存在着互相…

pip用法

pip show -f 包名 可以显示指定包的文件安装路径 参考&#xff1a; pip的基本使用 - 习久性成 - 博客园pip简介 1、pip 是一个现代的&#xff0c;通用的 Python 包管理工具。提供了对 Python 包的查找、下载、安装、卸载的功能。 2、pip是官方推荐的安装和管理Python包的工具…

进阶:Docker容器管理工具——Docker-Compose使用

文章目录 前言Compose大杀器编排服务 1、docker-compose安装curl方式安装增加可执行权限查看版本 2、Docker-compose.yaml命令3、 docker-compose实战4、Docker网络路由docker的跨主机网络路由**问题由来**:方案两台机分别配置路由表ip_forward配置 总结 前言 容器的管理工具&…

HarmonyOS NEXT,生命之树初长成

在不同的神话体系中&#xff0c;都有着关于生命之树的记载。 比如在北欧神话中&#xff0c;一株巨大的树木联结着九大世界&#xff0c;其被称为“尤克特拉希尔”Yggdrasill。在中国的《山海经》中&#xff0c;也有着“建木”的传说&#xff0c;它“有九欘&#xff0c;下有九枸&…

Squeeze-and-Excitation Networks阅读笔记一

文章目录 Abstract1 INTRODUCTION Abstract 卷积算子&#xff08;convolution operator&#xff09;是卷积神经网络&#xff08;cnn&#xff09;的核心组成部分&#xff0c;它使网络能够通过融合每层局部接受域内的空间和通道信息来构建信息特征。广泛的先前研究已经调查了这种…

24考研数据结构-图的存储结构邻接矩阵

目录 6.3 储存结构&#xff08;邻接表表示法&#xff09;1. 储存方式2. 结构3. 图的邻接表存储表示&#xff08;算法&#xff09;4. 结论5. 邻接矩阵和邻接表的对比邻接矩阵优点&#xff1a;缺点&#xff1a; 邻接表优点&#xff1a;缺点&#xff1a; 邻接矩阵与邻接表的关系 6…

iphone短信备份软件哪个比较好用?牛学长苹果数据管理工具是首选!

iphone短信备份软件哪个比较好用&#xff1f;说到iPhone备份&#xff0c;想必大家会下意识的想到iCloud和iTunes&#xff0c;这是苹果官方提供的两大特色服务&#xff0c;可以帮助我们备份苹果手机中的短信等多项数据&#xff0c;但由于它们都各有限制&#xff0c;只能同步整机…