Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provider org.apache.seatunnel.connectors.seatunnel.file.s3.sink.S3FileSink could not be instantiated #6676

Closed
2 of 3 tasks
guoyangjie opened this issue Apr 10, 2024 · 3 comments
Labels

Comments

@guoyangjie
Copy link

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

When we use the S3 protocol to synchronize CSV files from MinIO to ClickHouse, an exception is thrown. The source is S3, and the target is ClickHouse.

SeaTunnel Version

2.3.4

SeaTunnel Config

{
    "env": {
        "job.mode": "BATCH"
    },

    "source": [
        {
            "path": "/collect/test.csv",
            "bucket": "s3a://datamanager",
            "schema": {
                "fields": {
                    "ID": "int",
                    "NAME": "string"
                }
            },
            "access_key": "GMWCEab2XZNyINHb4nXM",
            "secret_key": "QPmEPsrZuacLiu8gq0zpUxe9QsetJGtrLvgytRWp",
            "plugin_name": "S3File",
            "read_columns": [
                "ID",
                "NAME"
            ],
            "fs.s3a.endpoint": "http://192.168.3.1:9000",
            "fs.s3a.aws.credentials.provider": "org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider",
            "file_format_type": "csv",
            "result_table_name": "sourceTable",
            "skip_header_row_number": 1
            
        }
    ],
    "transform": [
        {
            "query": "select ID as id,NAME as name from sourceTable  where 1=1 ",
            "plugin_name": "Sql",
            "result_table_name": "sinkTable",
            "source_table_name": "sourceTable"
        }
    ],
        "sink": [
        {
            "host": "192.168.3.1:8123",
            "table": "test_1",
            "database": "default",
            "password": "",
            "username": "default",
            "plugin_name": "Clickhouse",
            "source_table_name": "sinkTable"
        }
    ]
}

Running Command

seatunnel.cmd --config test.conf

Error Exception

java.util.ServiceConfigurationError: org.apache.seatunnel.api.sink.SeaTunnelSink: Provider org.apache.seatunnel.connectors.seatunnel.file.s3.sink.S3FileSink could not be instantiated
        at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[?:1.8.0_382]
        at java.util.ServiceLoader.access$100(ServiceLoader.java:185) ~[?:1.8.0_382]
        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) ~[?:1.8.0_382]
        at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_382]
        at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_382]
        at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.loadPluginInstance(AbstractPluginDiscovery.java:302) ~[seatunnel-starter.jar:2.3.4]
        at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.createOptionalPluginInstance(AbstractPluginDiscovery.java:183) ~[seatunnel-starter.jar:2.3.4]
        at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.createPluginInstance(AbstractPluginDiscovery.java:227) ~[seatunnel-starter.jar:2.3.4]
        at org.apache.seatunnel.engine.core.parse.ConnectorInstanceLoader.loadSinkInstance(ConnectorInstanceLoader.java:77) ~[seatunnel-starter.jar:2.3.4]
        at org.apache.seatunnel.engine.core.parse.JobConfigParser.parseSink(JobConfigParser.java:194) ~[seatunnel-starter.jar:2.3.4]
        at org.apache.seatunnel.engine.core.parse.JobConfigParser.parseSinks(JobConfigParser.java:170) ~[seatunnel-starter.jar:2.3.4]
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSink(MultipleTableJobConfigParser.java:531) ~[seatunnel-starter.jar:2.3.4]
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:193) ~[seatunnel-starter.jar:2.3.4]
        at org.apache.seatunnel.engine.server.rest.RestJobExecutionEnvironment.getLogicalDag(RestJobExecutionEnvironment.java:76) ~[seatunnel-starter.jar:2.3.4]
        at org.apache.seatunnel.en

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@guoyangjie guoyangjie added the bug label Apr 10, 2024
@pppeace
Copy link

pppeace commented Apr 10, 2024

same phenomenon.

@Hisoka-X
Copy link
Member

Hisoka-X commented Apr 10, 2024

please retry with dev branch. Already fixed by #6551

@guoyangjie
Copy link
Author

Thank you, I'll give it a try

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants