Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MongoDB - Sink - PluginIdentifier not found #6800

Open
2 of 3 tasks
a11dev opened this issue May 6, 2024 · 14 comments
Open
2 of 3 tasks

MongoDB - Sink - PluginIdentifier not found #6800

a11dev opened this issue May 6, 2024 · 14 comments
Labels
bug

Comments

@a11dev
Copy link

a11dev commented May 6, 2024

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

simple seatunnel configuration , designed to sink oracle to mongodb.
Oracle to postgres is working.
I added a new sink

	MongoDB {
		source_table_name = "source"
		uri = "mongodb://user:pwd@ipaddress:27017/dbname?readPreference=secondary&slaveOk=true"
		database = "dbname"
		collection = "destcollection"
		upsert-enable = true
		primary-key = ["pkname"]

	}

it ends with this exception.

Is related to mongodb java library, but which and where must it be installed.
I've done it but no solution I tried is working.

I tried with :

mongodb-driver-core-5.1.0
mongodb-java-driver-5.1.0

and as alternative:

mongodb-driver-sync-5.1.0

placing it into:
seatunnel_home\lib
plugins\MongoDB\lib
connectors\seatunnel

documentation link to the right download is failing.

SeaTunnel Version

2.3.4

SeaTunnel Config

#Also into the first attachment:


env {
	parallelism = 2
    job.mode=STREAMING
    job.name=SeaTunnel_Job
    read_limit.bytes_per_second=7000000
    read_limit.rows_per_second=400
}

  Oracle-CDC {

    result_table_name = "tab1"
    base-url = "jdbc:oracle:thin:user/password@ip:1521/service_name"
    source.reader.close.timeout = 120000
    username = "user"
    password = "password"
    database-names = ["DBNAME"]
	# real db name DBNAME.domain.local ( it works with DBNAME )
    schema-names = ["SCHEMA"]
    startup.mode = "INITIAL"
    table-names = ["DBNAME.SCHEMA.TABLE1"]
  }


}


sink {
	MongoDB {
		source_table_name = "source"
		uri = "mongodb://user:pwd@ipaddress:27017/dbname?readPreference=secondary&slaveOk=true"
		database = "dbname"
		collection = "destcollection"
		upsert-enable = true
		primary-key = ["pkname"]

	}

}



### Running Command

```shell
java -Dlog4j2.configurationFile=E:\programmi\apache-seatunnel-2.3.4\config\log4j2_client.properties -Dhazelcast.client.config=E:\programmi\apache-seatunnel-2.3.4\config\hazelcast-client.yaml -Dseatunnel.config=E:\programmi\apache-seatunnel-2.3.4\config\seatunnel.yaml -Dhazelcast.config=E:\programmi\apache-seatunnel-2.3.4\config\hazelcast.yaml -Dseatunnel.logs.file_name=seatunnel-starter-clienttest -Xms256m -Xmx512m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=E:\programmi\apache-seatunnel-2.3.4\dump\zeta-client  -cp E:\programmi\apache-seatunnel-2.3.4\lib\*;E:\programmi\apache-seatunnel-2.3.4\starter\seatunnel-starter.jar org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient  --config .\config\v2.batch.config.template -m local

Error Exception

2024-05-06 13:42:06,610 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Fatal Error, 

2024-05-06 13:42:06,611 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Please submit bug report in https://github.com/apache/seatunnel/issues

2024-05-06 13:42:06,612 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Reason:SeaTunnel job executed failed 

2024-05-06 13:42:06,614 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:199)
	at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
	at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: java.lang.RuntimeException: Plugin PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='MongoDB'} not found.
	at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.createPluginInstance(AbstractPluginDiscovery.java:231)
	at org.apache.seatunnel.engine.core.parse.ConnectorInstanceLoader.loadSinkInstance(ConnectorInstanceLoader.java:77)
	at org.apache.seatunnel.engine.core.parse.JobConfigParser.parseSink(JobConfigParser.java:194)
	at org.apache.seatunnel.engine.core.parse.JobConfigParser.parseSinks(JobConfigParser.java:170)
	at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSink(MultipleTableJobConfigParser.java:531)
	at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:193)
	at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
	at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:161)
	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:146)
	... 2 more

Zeta or Flink or Spark Version

Zeta

Java or Scala Version

JAva

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@a11dev a11dev added the bug label May 6, 2024
@Hisoka-X
Copy link
Member

Hisoka-X commented May 9, 2024 •
edited
Loading

Could you try with 2.3.5? It should be fixed by #6551

@a11dev
Copy link
Author

a11dev commented May 9, 2024 via email

@Hisoka-X
Copy link
Member

Hisoka-X commented May 9, 2024

You can get download link from https://www.apache.org/dyn/closer.lua/seatunnel/2.3.5/apache-seatunnel-2.3.5-bin.tar.gz

@a11dev
Copy link
Author

a11dev commented May 9, 2024 via email

@Hisoka-X
Copy link
Member

Hisoka-X commented May 9, 2024

did you execute bin/install-plugin.sh ? https://seatunnel.apache.org/docs/2.3.5/start-v2/locally/deployment#step-3-install-connectors-plugin

@a11dev
Copy link
Author

a11dev commented May 13, 2024 via email

@Hisoka-X
Copy link
Member

cc @liugddx

@a11dev
Copy link
Author

a11dev commented Jun 24, 2024 via email

@a11dev
Copy link
Author

a11dev commented Jun 24, 2024 via email

@Hisoka-X
Copy link
Member

Is there a way to update a collection in order to perform a dernomalization along the oracle to mongo sync?

not sure what you want, could you share some example?

@a11dev
Copy link
Author

a11dev commented Jun 25, 2024 via email

@Hisoka-X
Copy link
Member

Sorry, seatunnel can not do this at now.

@a11dev
Copy link
Author

a11dev commented Jun 25, 2024 via email

@a11dev
Copy link
Author

a11dev commented Jun 25, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Hisoka-X @a11dev