您现在的位置是:首页 >技术杂谈 >flink cdc 2.3.0 Caused by: java.lang.NoSuchMethodError: io.debezium.connector.mysql.MySqlConnection网站首页技术杂谈
flink cdc 2.3.0 Caused by: java.lang.NoSuchMethodError: io.debezium.connector.mysql.MySqlConnection
在Flink 1.14 中 使用 flink-connector-mysql-cdc 2.3.0的依赖,里面默认引用的debezium-connector-mysql 的 debezium.version 是1.6.4.Final 版本的,在将任务提交到集群时报错了
Caused by: java.lang.NoSuchMethodError: io.debezium.connector.mysql.MySqlConnection$MySqlConnectionConfiguration.<init>(Lio/debezium/config/Configuration;Ljava/util/Properties;)V at
全文报错信息如下:
2025-02-07 20:48:28.727 [Source: TableSourceScan(table=[[default_catalog, default_database, orders]], fields=[order_id, order_date, customer_name, price, product_id, order_status]) (1/1)#2] WARN org.apache.flink.runtime.taskmanager.Task - Source: TableSourceScan(table=[[default_catalog, default_database, orders]], fields=[order_id, order_date, customer_name, price, product_id, order_status]) (1/1)#2 (f283d120ff7c065dbef7646fcd5a4b6e) switched from RUNNING to FAILED with failure cause: java.lang.RuntimeException: One or more fetchers have encountered exception at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:225) at org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:169) at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:130) at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:354) at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:495) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:806) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:758) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.NoSuchMethodError: io.debezium.connector.mysql.MySqlConnection$MySqlConnectionConfiguration.<init>(Lio/debezium/config/Configuration;Ljava/util/Properties;)V at com.ververica.cdc.connectors.mysql.debezium.DebeziumUtils.createMySqlConnection(DebeziumUtils.java:78) at com.ververica.cdc.connectors.mysql.debezium.DebeziumUtils.createMySqlConnection(DebeziumUtils.java:71) at com.ververica.cdc.connectors.mysql.source.reader.MySqlSplitReader.checkSplitOrStartNext(MySqlSplitReader.java:138) at com.ververica.cdc.connectors.mysql.source.reader.MySqlSplitReader.fetch(MySqlSplitReader.java:71) at org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:58) at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:142) at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:105) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more
原因是 MySqlConnection 这个类 在flink-connector-mysql-cdc 2.3.0 和 debezium-connector-mysql 1.6.4.Final中都存在,这两个包路径一样 都是 io.debezium.connector.mysql,JVM在编译的时候默认优先找debezium-connector-mysql的,这里编译就会有问题
解决方案是去flink服务 的lib文件夹下
1、把 debezium-connector-mysql-1.6.4.Final.jar 改名,比如 flink-dep-debezium-connector-mysql-1.6.4.Final.jar ,目的是让 JVM 优先找到 flink cdc 内与 debezium 同包同名的类。
2、把 flink-connector-mysql-cdc-2.3.0.jar 内的 debezium 包覆盖到 debezium-connector-mysql-1.6.4.Final.jar
亲测第一种方案可行,可以解决这个报错
本人理解第二种方案是把flink-connector-mysql-cdc-2.3.0.jar里面 io.debezium包路径的类把debezium-connector-mysql-1.6.4.Final.jar里面同样包路径的类都覆盖,然后重新打包,有感兴趣的可以试一下,看是否可行
这个错误纠结了两天都没搞定,后来在github的issues里面找到的这个解决方案,为了让大家快点解决这个问题,写下来供大家参考