【数据同步】SeaTunnel初体验,5000字深入浅出带你用上Oracle-CDC

news2025/1/20 5:53:06

Apache SeaTunnel 是啥?
下一代高性能、分布式、海量数据集成框架。
支持上百个数据源、传输速度快、准确率高,丰富易扩展的连接器和插件化的连接器设计,能够更轻松的运行复杂的集成。
是一个分布式、高性能的数据集成平台,用于数据迁移和实时流处理任务。

JDK下载安装

需安装Java (Java 8 或 11, 其他高于Java 8的版本理论上也可以工作) 以及设置 JAVA_HOME

本次选择了安装8u381版本,大部分的bug有得到修复,JDK下载页地址: https://www.oracle.com/java/technologies/javase/javase8u211-later-archive-downloads.html

# 解压
tar xvf jdk-8u381-linux-x64.tar.gz
# 配置环境变量,添加2行
cat ~/.bash_profile

export JAVA_HOME="/data/jdk-8u381-linux-x64"
export PATH=$PATH:$JAVA_HOME/bin

# 环境变量生效
source ~/.bash_profile

# 验证
java -version

java version "1.8.0_381"
Java(TM) SE Runtime Environment (build 1.8.0_381-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.381-b09, mixed mode)

SeaTunnel 下载安装

其实安装很简单,下载解压,开箱即用,截止作者撰写本文的时候,最新版本2.3.5,所以本文的下载链接均为2.3.5版本,目前社区最新版本是2.3.7

官网下载安装包:[bin] apache-seatunnel-2.3.5-bin.tar.gz

参考文档

  • https://seatunnel.apache.org/zh-CN/docs/2.3.5/start-v2/locally/deployment
    • https://seatunnel.apache.org/zh-CN/docs/2.3.5/seatunnel-engine/deployment

下载及解压

file

cd /data/
export version="2.3.5"
wget "https://archive.apache.org/dist/seatunnel/${version}/apache-seatunnel-${version}-bin.tar.gz"
tar -xzvf "apache-seatunnel-${version}-bin.tar.gz"

配置环境变量

cat  /etc/profile.d/seatunnel.sh

export SEATUNNEL_HOME=/data/apache-seatunnel-2.3.5
export PATH=$PATH:$SEATUNNEL_HOME/bin

# 环境变量生效
source /etc/profile.d/seatunnel.sh

安装连接器

配置插件文件(按需)

连接器数据量很多,全部下载需要时间及耗费存储空间,本次仅添加了Oracle-cdc 和一些常用的连接器,配置文件如下

cat  config/plugin_config

--connectors-v2--
connector-cdc-mysql
connector-cdc-mongodb
connector-cdc-oracle
connector-elasticsearch
connector-http-wechat
connector-jdbc
connector-kafka
connector-mongodb
connector-redis
--end--

自动安装连接器插件(指定版本)

这里跟大家讲一下,从2.2.0-beta版本开始,二进制包不再默认提供连接器依赖,第一次使用需执行如下命令安装连接器。

若手动安装,安装包地址:https://repo.maven.apache.org/maven2/org/apache/seatunnel/

# 2.3.5 为当前安装的版本
sh bin/install-plugin.sh 2.3.5

安装比较耗时,配置的越多越耗时,如下展示部分日志

Install SeaTunnel connectors plugins, usage version is 2.3.5
install connector :  connector-cdc-oracle
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< org.apache.maven:standalone-pom >-------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:get (default-cli) @ standalone-pom ---
[WARNING] The artifact xml-apis:xml-apis:jar:2.0.2 has been relocated to xml-apis:xml-apis:jar:1.0.b2
[INFO] Resolving org.apache.seatunnel:connector-cdc-oracle:jar:2.3.5 with transitive dependencies
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/seatunnel/connector-cdc-oracle/2.3.5/connector-cdc-oracle-2.3.5.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/seatunnel/connector-cdc-oracle/2.3.5/connector-cdc-oracle-2.3.5.pom (3.2 kB at 2.3 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/seatunnel/connector-cdc-oracle/2.3.5/connector-cdc-oracle-2.3.5.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/seatunnel/connector-cdc-oracle/2.3.5/connector-cdc-oracle-2.3.5.jar (31 MB at 411 kB/s)
[WARNING] destination/dest parameter is deprecated: it will disappear in future version.
[INFO] Copying /root/.m2/repository/org/apache/seatunnel/connector-cdc-oracle/2.3.5/connector-cdc-oracle-2.3.5.jar to /data/apache-seatunnel-2.3.5/connectors
[WARNING] Notice transitive dependencies won't be copied.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:17 min
[INFO] Finished at: 2024-07-25T11:34:52+08:00
[INFO] ------------------------------------------------------------------------

## 过程省略一万字。。 ##

安装完成后,连接器存放目录为: $SEATUNNEL_HOME/connectors/ 【重要】

安装完成后,检查连接器安装情况

file

ORACLE CDC连接测试

这里以11G为例:

安装JDBC驱动

下载地址: https://mvnrepository.com/artifact/com.oracle.database.jdbc/ojdbc8/12.2.0.1

file

JDBC驱动下载后存放目录:$SEATUNNEL_HOME/lib

源库相关配置

这里需要重启,生产环境谨慎操作!!

开启归档及补充日志(注意查看磁盘空间,避免爆盘事件)

ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;
alter system set log_archive_format='ARC%S_%R.%T.arc';
shutdown immediate
startup mount;
alter database archivelog;
alter database open;

源库创建表空间、用户,及授权

-- 创建用户及表空间
CREATE TABLESPACE logminer_tbs DATAFILE '/mnt/oracle/oradata/logminer_tbs.dbf' SIZE 25M AUTOEXTEND ON NEXT 512M MAXSIZE 32767M;
CREATE USER LOGMINER_USER IDENTIFIED BY Vir123 DEFAULT TABLESPACE logminer_tbs QUOTA UNLIMITED ON logminer_tbs;

-- 给用户授权
GRANT CREATE SESSION TO logminer_user;
GRANT SELECT ON V_$DATABASE to logminer_user;
GRANT SELECT ON V_$LOG TO logminer_user;
GRANT SELECT ON V_$LOGFILE TO logminer_user;
GRANT SELECT ON V_$LOGMNR_LOGS TO logminer_user;
GRANT SELECT ON V_$LOGMNR_CONTENTS TO logminer_user;
GRANT SELECT ON V_$ARCHIVED_LOG TO logminer_user;
GRANT SELECT ON V_$ARCHIVE_DEST_STATUS TO logminer_user;
GRANT EXECUTE ON DBMS_LOGMNR TO logminer_user;
GRANT EXECUTE ON DBMS_LOGMNR_D TO logminer_user;

-- 授权表权限
GRANT SELECT ANY TABLE TO logminer_user;
GRANT ANALYZE ANY TO logminer_user;

-- 增加权限,解决权限不足问题
GRANT SELECT ANY TRANSACTION TO logminer_user; 
GRANT SELECT ON V_$TRANSACTION TO logminer_user;

SeaTunnel 增加作业配置文件

配置文件目录: $SEATUNNEL_HOME/config

cd $SEATUNNEL_HOME
cat config/oracdc_console

env {
  # You can set engine configuration here
  parallelism = 1
  job.mode = "STREAMING"
  checkpoint.interval = 5000
}

source {
  # This is a example source plugin **only for test and demonstrate the feature source plugin**
  Oracle-CDC {
    driver = "oracle.jdbc.driver.OracleDriver"
    result_table_name = "customers"
    username = "LOGMINER_USER"
    password = "Vir123"
    database-names = ["VIRDB"]
    schema-names = ["VUSER"]
    table-names = ["VIRDB.VUSER.CT_DB_LABORATORY_BAK"]
    base-url = "jdbc:oracle:thin:@10.10.10.10:1521/VIRDB"
    source.reader.close.timeout = 120000
    connection.pool.size = 1
  }
}

transform {
}

sink {
    Console {
        source_table_name = "customers"
      }
}

本地启动作业,查看日志情况

sh ./bin/seatunnel.sh -e local --config ./config/oracdc_console

file

源端插入一条数据后,日志更新

file

汇总信息可以看到,总数从11条变成12条

file

常见问题

作业启动报错

报错信息为:Unable to create a source for identifier 'Jdbc'

===============================================================================
2024-07-24 14:38:08,672 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Fatal Error,

2024-07-24 14:38:08,673 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Please submit bug report in https://github.com/apache/seatunnel/issues

2024-07-24 14:38:08,673 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Reason:SeaTunnel job executed failed

2024-07-24 14:38:08,674 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a source for identifier 'Jdbc'.
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:100)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:332)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:188)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:156)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:149)
        ... 2 more
Caused by: org.apache.seatunnel.api.table.catalog.exception.CatalogException: ErrorCode:[API-03], ErrorDescription:[Catalog initialize failed] - Failed connecting to jdbc:oracle:thin:@10.10.100.140:1521:jszx via JDBC.
        at org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.AbstractJdbcCatalog.getConnection(AbstractJdbcCatalog.java:123)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.AbstractJdbcCatalog.open(AbstractJdbcCatalog.java:129)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.utils.JdbcCatalogUtils.getTables(JdbcCatalogUtils.java:78)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSource.<init>(JdbcSource.java:57)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSourceFactory.lambda$createSource$0(JdbcSourceFactory.java:78)
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:112)
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:73)
        ... 7 more
Caused by: java.sql.SQLException: No suitable driver found for jdbc:oracle:thin:@10.10.100.140:1521:jszx
        at java.sql.DriverManager.getConnection(DriverManager.java:689)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.AbstractJdbcCatalog.getConnection(AbstractJdbcCatalog.java:119)
        ... 13 more

解决方法:

(1) 驱动配置错误

正确的应该是oracle.jdbc.driver.OracleDriver 而非oracle.jdbc.OracleDriver

(2) 驱动错误问题

之前用了ojdbc14-10.2.0.3.0.jar 支持不了,重新下载了JDBC的驱动,如上文提到的“安装Jdbc驱动”

找不到ORACLE-CDC 驱动

===============================================================================


2024-07-24 17:15:37,657 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Fatal Error,

2024-07-24 17:15:37,657 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Please submit bug report in https://github.com/apache/seatunnel/issues

2024-07-24 17:15:37,657 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Reason:SeaTunnel job executed failed

2024-07-24 17:15:37,661 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: java.lang.RuntimeException: Plugin PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Oracle-CDC'} not found.
        at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.createPluginInstance(AbstractPluginDiscovery.java:234)
        at org.apache.seatunnel.engine.core.parse.ConnectorInstanceLoader.loadSourceInstance(ConnectorInstanceLoader.java:61)
        at org.apache.seatunnel.engine.core.parse.JobConfigParser.parseSource(JobConfigParser.java:77)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:327)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:188)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:156)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:149)
        ... 2 more

2024-07-24 17:15:37,661 ERROR [o.a.s.c.s.SeaTunnel           ] [main] -

解决方法:

重点关注

“Caused by: java.lang.RuntimeException: Plugin PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Oracle-CDC'} not found.”

报错找不到ORACLE-CDC ,可以通过配置文件 $SENTUNNEL_HOME/connectors/plugin-mapping.properties,查看当前支持的连接器名称,再配置后进行安装!

cat $SENTUNNEL_HOME/config/plugin_config

--connectors-v2--
connector-cdc-oracle
--end--

再次运行连接器插件安装
sh bin/install-plugin.sh 2.3.5

权限问题

ORA-01031: insufficient privileges

===============================================================================



Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.RuntimeException: One or more fetch                       ers have encountered exception
        at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherM                       anager.java:147)
        at org.apache.seatunnel.connectors.seatunnel.common.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:1                       60)
        at org.apache.seatunnel.connectors.seatunnel.common.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:93)
        at org.apache.seatunnel.connectors.cdc.base.source.reader.IncrementalSourceReader.pollNext(IncrementalSourceReader.java                       :111)
        at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:156)
        at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:116)
        at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168)
        at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:121)
        at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:703)
        at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1004)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records
        at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:165)
        at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:81)
        ... 5 more
Caused by: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector                        will be stopped.
        at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)
        at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.                       java:341)
        at org.apache.seatunnel.connectors.seatunnel.cdc.oracle.source.reader.fetch.logminer.OracleRedoLogFetchTask$RedoLogSpli                       tReadTask.execute(OracleRedoLogFetchTask.java:130)
        at org.apache.seatunnel.connectors.seatunnel.cdc.oracle.source.reader.fetch.logminer.OracleRedoLogFetchTask.execute(Ora                       cleRedoLogFetchTask.java:73)
        at org.apache.seatunnel.connectors.cdc.base.source.reader.external.IncrementalSourceStreamFetcher.lambda$submitTask$0(I                       ncrementalSourceStreamFetcher.java:106)
        ... 5 more
Caused by: java.sql.SQLSyntaxErrorException: ORA-01031: insufficient privileges

        at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:494)
        at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:446)
        at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1054)
        at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:623)
        at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
        at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:612)
        at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:226)
        at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:59)
        at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:747)
        at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:904)
        at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1082)
        at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3780)
        at oracle.jdbc.driver.T4CPreparedStatement.executeInternal(T4CPreparedStatement.java:1343)
        at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3822)
        at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1165)
        at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.                       java:291)
        ... 8 more

解决方法: 增加TRANSACTION的权限

GRANT SELECT ANY TRANSACTION TO logminer_user; 
GRANT SELECT ON V_$TRANSACTION TO logminer_user;

总的来说,SeaTunnel的安装配置还是很简单的,如果你是初次接触,出现了各种文档不熟悉,各部件不熟悉漏了下载的情况,以及各个JAR包、Connector的位置存放以及配置混乱的等问题,可以通过社区Github、邮件列表及微信社群来进行解决,包括查看社区的文章及参加社区的活动!

最后也感谢Apache SeaTunnel社群支持的小伙伴们!

参考文档
  • 下载地址:https://seatunnel.apache.org/zh-CN/download
  • 文档参考:https://seatunnel.apache.org/zh-CN/docs/2.3.5/connector-v2/source/Oracle-CDC
  • 源码查询: https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-oracle-e2e
  • issue记录: https://github.com/apache/seatunnel/issues/6799

本文由 白鲸开源科技 提供发布支持!

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.coloradmin.cn/o/2059437.html

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈,一经查实,立即删除!

相关文章

USB Type-C如何取9V、12V、15V、20V电压-PD快充协议芯片ECP5701

相信大家在生活中也发现了&#xff0c;现在越来越多的设备都改用这种type-C接口的母座进行取电了。 因为欧盟决议 &#xff1a;自2024年起部分消费电子产品必须提供单一的USB-C充电接口。 那么这种type-C接口相比之前的Micro-B接口有着一个很大的优势就是可以有更高的电压&…

部署同步工具syncthing

1、下载包arm包&#xff08;根据自己的环境下载包&#xff09; #进到指定目录 cd /usr/local/ #可以根据自己的环境下载不同版本的包 wget https://github.com/syncthing/syncthing/releases/download/v1.27.10/syncthing-linux-arm64-v1.27.10.tar.gz2、进行部署 #将其解压 …

接口参数与文档||关于淘宝商品·订单数据API接口的功能达成经验分享

电商数据采集有5种方式&#xff0c;包括API、RPA、数据库连接、Excel下载和ERP等业务系统数据采集。这些方法可帮助卖家获取多平台电商数据&#xff0c;进行深度挖掘&#xff0c;实现电商运营的优化。 电商竞争白热化的今天&#xff0c;一个电商卖家往往会在多个平台铺设店铺来…

Flutter Web 正式官宣弃用 HTML renderer , Canvas 路线成为唯一

Flutter Web 团队计划在 2025 年的第一个 Flutter stable 版本中弃用 HTML renderer&#xff0c;当然在 master 和 beta 中会更早合并这一更改。 关于这个话题&#xff0c;其实在年初的我就曾发布过 《Flutter 即将放弃 Html renderer 》&#xff0c; Html renderer 从 2018 年…

狗都能看懂的Swin Transformer的讲解和代码实现

文章目录 1、Swin-Transformer介绍2、模型整体框架3、Patch Mergeing详解4、W-MSA模块详解MSA模块计算量W-MSA模块计算量 5、SW-MSA详解6、Relative Position Bias详解7、模型详细配置参数 1、Swin-Transformer介绍 自从ViT&#xff08;Vision Transformer&#xff09;出现之后…

腾讯软件测试岗二面:web 测试问题被虐哭了,直到学长给了我这些知识点.....

web 测试一直是大厂软件测试问到的一个重点&#xff0c;下面给大家展示下大厂关于web 测试经常会问到的一些问题&#xff0c;以及解析。看当面试官问到你这些问题的时候&#xff0c;你是否也能够对答如流。 web 测试面试真题及解析&#xff1a; 一&#xff0c;描述用浏览器访问…

SQL基础——函数与约束

声明&#xff1a;以下内容为根据黑马数据库视频教程&#xff0c;个人整理的笔记&#xff0c;方便记录学习。 SQL基础之函数与约束 知识导图案例SQL语句编写一、函数1.字符串函数2.数值函数3.日期函数4.流程函数 二、约束 知识导图 案例SQL语句编写 一、函数 1.字符串函数 A.…

用工业操作系统鸿道Intewell可以玩黑神话悟空吗?

黑神话悟空的爆火&#xff0c;让我想到&#xff0c;工业操作系统鸿道Intewell是否可以玩黑神话悟空&#xff1f; 鸿道Intewell操作系统是面向工业控制领域的操作系统&#xff0c;它支持实时和非实时应用在同一个硬件平台上运行&#xff0c;并且能够保留Windows开发环境的同时部…

平衡二叉树(AVLTree)

1.平衡二叉树的定义 1.1 什么是平衡二叉树 平衡二叉树&#xff0c;又称AVL树&#xff0c;用于解决二叉排序树高度不确定的情况&#xff0c;如果二叉排序树的子树间的高度相差太大&#xff0c;就会让二叉排序树操作的时间复杂度升级为O(n)&#xff0c;为了避免这一情况&#x…

mac 安装Arthas

mac安装有两种方式 1.第一步安装Arthas 第一种&#xff1a; curl -L https://arthas.aliyun.com/install.sh | sh 第二种jar包形式 curl -O https://arthas.aliyun.com/arthas-boot.jar个人比较推荐第一种因为运行测试成功了 第一种安装后可能会出现一些命令不符合 需…

NVIDIA刚刚发布了关于 AI 的免费在线课程!

英伟达最近上线了一批不错的免费课程&#xff0c;先收藏起来&#xff01; ps:有时候真想有一个收藏即学会的技能啊 数据中心中的人工智能 了解数据中心的 AI 基础知识&#xff0c;涵盖机器学习、深度学习、GPU 架构和部署。 了解多系统 AI 集群和基础设施规划。 课程地址&…

勇闯机器学习(第三关-特征工程)

以下内容皆为原创&#xff0c;制作不易&#xff0c;请帅锅、镁铝点点赞赞和关注吧❥(^_^) 一.提问环节 机器学习是什么&#xff1f; 机器学习就是通过自动分析大量数据去建立模型&#xff0c;训练模型&#xff0c;预测数据。 这么好记的概念&#xff0c;你应该记住了吧&#x…

多线程编程的拙见

一. 线程和进程的概念 1.为什么引入多线程编程&#xff1f; 在多线程&#xff08;Multithreaded&#xff0c;MT&#xff09;编程出现之前&#xff0c;电脑程序的运行由一个执行序列组成&#xff0c;执行序列按顺序在主机的中央处理器CPU中运行。即使整个程序由多个相互独立无…

基于Yolov5的安全帽检测系统设计与开发(论文+源码)_kaic

摘 要 安全帽检测系统的设计意义在于提高工作场所的安全性和生产效率&#xff0c;通过安全帽检测系统可以实时监控工人是否佩戴安全帽&#xff0c;及时发现不佩戴安全帽的工人并进行提醒和警示&#xff0c;避免因为不佩戴安全帽导致意外事故的发生。因此&#xff0c;本文是一个…

云呼叫中心系统哪家好?这个系统强烈推荐

云呼叫中心系统作为企业客户服务与运营的核心平台&#xff0c;正逐步成为提升企业竞争力的关键要素。面对市场上琳琅满目的云呼叫中心系统提供商&#xff0c;企业往往面临选择难题&#xff1a;“云呼叫中心系统哪家好&#xff1f;” 云呼叫中心系统&#xff1a;定义与核心功能 …

【HuggingFace Transformers】BertModel源码解析

BertModel源码解析 1. BertModel 介绍2. BertModel 源码逐行注释 1. BertModel 介绍 BertModel 是 transformers 库中的核心模型之一&#xff0c;它实现了 BERT&#xff08;Bidirectional Encoder Representations from Transformers&#xff09;模型的架构。BERT 是基于 Trans…

百数教学——企业互联,让业务合作跨越距离与障碍

一、简介 企业互联作为强大的纽带&#xff0c;不仅促进了企业间的紧密合作与深入交流&#xff0c;还显著提升了企业的运营效率和市场竞争力。它通过无缝链接外部组织&#xff0c;实现双方企业项目数据的即时共享与整合&#xff0c;从而深化了业务合作的层次与广度&#xff0c;…

如何在SOLIDWORKS中自定义材料库

使用SolidWorks对模型进行评估时都需要现定义一个材料&#xff1b;例如&#xff1a;测量模型的质量属性、使用限元分析模型等&#xff0c;SolidWorks自带了丰富的材质库&#xff0c;满足了绝大部分的设计需求。但是难免会遇到一些特点的材质属性&#xff0c;在SOLIDWORKS中无法…

【LLM大模型论文日更】| 格式胜过内容:揭示大型语言模型的提示效应

论文&#xff1a;https://arxiv.org/pdf/2408.08780代码&#xff1a;未开源机构&#xff1a;北京大学领域&#xff1a;提示学习&#xff0c;上下文学习发表&#xff1a;arXiv 研究背景 研究问题&#xff1a;这篇文章探讨了在上下文学习&#xff08;ICL&#xff09;中&#xf…

ubuntu20.4 安装postgresql并远程连接

1、安装对应的库 如果提示安装失败,E: Unable to locate package postgresql这个错误的话,那就更新下,执行sudo apt update 命令,再次下载,应该就可以成功 sudo apt install postgresql 2、端口设置 检查5432端口是否监听 ss -nlt 需要将多个远程客户端连接上数据库,配…