[Bug] [deploy on k8s] SPRING_DATASOURCE_DRIVER_CLASS_NAME need add in templates/_helpers.tpl #10103

zwZjut · 2022-05-18T10:45:51Z

Search before asking

I had searched in the issues and found no similar issues.

What happened

[Bug] [deploy on k8s] SPRING_DATASOURCE_DRIVER_CLASS_NAME need add in templates/_helpers.tpl

What you expected to happen

[Bug] [deploy on k8s] SPRING_DATASOURCE_DRIVER_CLASS_NAME need add in templates/_helpers.tpl

How to reproduce

[Bug] [deploy on k8s] SPRING_DATASOURCE_DRIVER_CLASS_NAME need add in templates/_helpers.tpl

Anything else

[Bug] [deploy on k8s] SPRING_DATASOURCE_DRIVER_CLASS_NAME need add in templates/_helpers.tpl

Version

dev

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

zwZjut · 2022-05-18T10:47:03Z

add in values.yaml;
postgresql:
driver: org.postgresql.Driver

add in templates/_helpers.tpl ;

name: SPRING_DATASOURCE_DRIVER_CLASS_NAME
{{- if .Values.postgresql.enabled }}
value: {{ .Values.postgresql.driver }}
{{- else }}
value: {{ .Values.externalDatabase.driver }}
{{- end }}

github-actions · 2022-05-18T11:15:27Z

Thank you for your feedback, we have received your issue, Please wait patiently for a reply.

In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
If you haven't received a reply for a long time, you can join our slack and send your question to channel #troubleshooting

qingwli · 2022-05-19T05:49:36Z

Hi @zwZjut, I don't think we need to SPRING_DATASOURCE_DRIVER_CLASS_NAME.

Here is the reason.

You can see this define the DATABASE

And when you build the Docker image, will run the ./bin/start.sh on Api || Matset || Worker || Alert-Server.

The ./bin/start.sh will source $DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh

And dolphinscheduler_env.sh will set the SPRING_PROFILES_ACTIVE , you can see the code here

Then you can see the driver-class-name will be chosen from the different data sources. postgresql and mysql

qingwli · 2022-05-19T08:57:42Z

I have checked the yaml about Api || Matset || Worker || Alert-Server || tools, and they are two bugs.

tools module changes the this but not changed the yaml.

When you change the data source to Mysql but will still use data source postgresql

Woker module yaml does not contains Mysql config

qingwli · 2022-05-19T09:24:06Z

And I have another question about running the Docker image.

$ docker run -d --name dolphinscheduler-tools \
    -e DATABASE="postgresql" \
    # Use "com.mysql.cj.jdbc.driver" if you use MySQL
    -e SPRING_DATASOURCE_DRIVER_CLASS_NAME="org.postgresql.Driver" \
    -e SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/<DATABASE>" \
    -e SPRING_DATASOURCE_USERNAME="<USER>" \
    -e SPRING_DATASOURCE_PASSWORD="<PASSWORD>" \
    --net host \
    apache/dolphinscheduler-tools:"${DOLPHINSCHEDULER_VERSION}" tools/bin/upgrade-schema.sh

Do we need to config -e SPRING_DATASOURCE_DRIVER_CLASS_NAME="org.postgresql.Driver" \ ?

The reason please look commented above

WDYT @SbloodyS @caishunfeng

SbloodyS · 2022-05-19T09:32:04Z

And I have another question about running the Docker image.

$ docker run -d --name dolphinscheduler-tools \
    -e DATABASE="postgresql" \
    # Use "com.mysql.cj.jdbc.driver" if you use MySQL
    -e SPRING_DATASOURCE_DRIVER_CLASS_NAME="org.postgresql.Driver" \
    -e SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/<DATABASE>" \
    -e SPRING_DATASOURCE_USERNAME="<USER>" \
    -e SPRING_DATASOURCE_PASSWORD="<PASSWORD>" \
    --net host \
    apache/dolphinscheduler-tools:"${DOLPHINSCHEDULER_VERSION}" tools/bin/upgrade-schema.sh

Do we need to config -e SPRING_DATASOURCE_DRIVER_CLASS_NAME="org.postgresql.Driver" \ ?

The reason please look commented above

WDYT @SbloodyS @caishunfeng

Yes. It's needed.

kezhenxu94 · 2022-05-20T09:12:32Z

And I have another question about running the Docker image.

$ docker run -d --name dolphinscheduler-tools \
    -e DATABASE="postgresql" \
    # Use "com.mysql.cj.jdbc.driver" if you use MySQL
    -e SPRING_DATASOURCE_DRIVER_CLASS_NAME="org.postgresql.Driver" \
    -e SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/<DATABASE>" \
    -e SPRING_DATASOURCE_USERNAME="<USER>" \
    -e SPRING_DATASOURCE_PASSWORD="<PASSWORD>" \
    --net host \
    apache/dolphinscheduler-tools:"${DOLPHINSCHEDULER_VERSION}" tools/bin/upgrade-schema.sh

Do we need to config -e SPRING_DATASOURCE_DRIVER_CLASS_NAME="org.postgresql.Driver" \ ?

SPRING_DATASOURCE_DRIVER_CLASS_NAME shouldn't be specified, DATABASE is aimed at configuring the database related info, when users set DATABASE=postgresql the driver should be set automatically

kezhenxu94 · 2022-05-20T09:13:52Z

@liqingwang please add a profile postgresql here and set correct driver in the yaml

dolphinscheduler/dolphinscheduler-tools/src/main/resources/application.yaml

Lines 39 to 46 in e2fac21

    
           --- 
        
           spring: 
        
             config: 
        
               activate: 
        
                 on-profile: mysql 
        
             datasource: 
        
               driver-class-name: com.mysql.jdbc.Driver 
        
               url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8

 --- 
 spring: 
   config: 
     activate: 
       on-profile: postgresql 
   datasource: 
     driver-class-name: org.postgresql.Driver

qingwli · 2022-05-20T09:19:29Z

I agree.

If we don't need to specify the SPRING_DATASOURCE_DRIVER_CLASS_NAME, then we don't need this field when you start at Docker mode or K8s mode?

So this code doesn't need? @kezhenxu94

kezhenxu94 · 2022-05-20T09:21:57Z

I agree.

If we don't need to specify the SPRING_DATASOURCE_DRIVER_CLASS_NAME, then we don't need this field when you start at Docker mode or K8s mode?

So this code doesn't need? @kezhenxu94

Yes, we'd better keep the configurations minimal whenever they can be specified automatically, so users won't be overwhelming by so many configurations.

qingwli · 2022-05-20T09:23:25Z

And we can't delete source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh" in docker mode, this sh will export SPRING field, am I correct?

qingwli · 2022-05-20T09:27:14Z

Is my understanding of the driver set automatically correct? #10103 (comment)

kezhenxu94 · 2022-05-20T09:28:52Z

Is my understanding of the driver set automatically correct? #10103 (comment)

This is right

kezhenxu94 · 2022-05-20T09:32:29Z

And we can't delete source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh" in docker mode, this sh will export SPRING field, am I correct?

Hi @liqingwang , you don't need to delete source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh", if you set any env var in Kubernetes, it will override dolphinscheduler_env.sh, did you find anything different?

qingwli · 2022-05-20T09:35:09Z

Yes, but only one difference, we don't define SPRING_DATASOURCE_DRIVER_CLASS_NAME in Kubernetes config file, but I define the database.

And source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh will export SPRING_PROFILES_ACTIVE=${DATABASE} , this will change the database driver, so we need this sh

kezhenxu94 · 2022-05-20T09:36:51Z

Yes, but only one difference, we don't define SPRING_DATASOURCE_DRIVER_CLASS_NAME in Kubernetes config file, but I define the database.

And source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh will export SPRING_PROFILES_ACTIVE=${DATABASE} , this will change the database driver, so we need this sh

right

qingwli · 2022-05-20T09:37:41Z

I will follow the previous design pattern to change the code.

So here is another question. Pls see this #10128 (comment) , the active profile is to choose the database config, when we define the active profiles, and tools module can't change driver automatically. This is a bug. @SbloodyS

SbloodyS · 2022-05-20T09:39:46Z

I will follow the previous design pattern to change the code.

So here is another question. Pls see this #10128 (comment) , the active profile is to choose the database config, when we define the active profiles, and tools module can't change driver automatically. This is a bug. @SbloodyS

Yes.

kezhenxu94 · 2022-05-20T09:40:54Z

I will follow the previous design pattern to change the code.

So here is another question. Pls see this #10128 (comment) , the active profile is to choose the database config, when we define the active profiles, and tools module can't change driver automatically. This is a bug. @SbloodyS

As I said here #10103 (comment), can you update this in #10128 and see whether that satisfy your requirement?

qingwli · 2022-05-20T09:44:46Z

I'll have a try, thx @kezhenxu94

qingwli · 2022-05-20T09:50:17Z

I am not sure If I use java -Dspring.profiles.active=upgrade to define the profile java used, and then export SPRING_PROFILES_ACTIVE=mysql, Spring will use both of them, or MySQL will overwritten upgrade

kezhenxu94 · 2022-05-20T10:53:19Z

I am not sure If I use java -Dspring.profiles.active=upgrade to define the profile java used, and then export SPRING_PROFILES_ACTIVE=mysql, Spring will use both of them, or MySQL will overwritten upgrade

It will be overridden

qingwli · 2022-05-24T05:29:14Z

I used -Dspring.profiles.active=upgrade,${DATABASE} to use two profiles and the driver will be set automatically

zwZjut added bug Something isn't working Waiting for reply Waiting for reply labels May 18, 2022

songjianet assigned songjianet and unassigned songjianet May 18, 2022

songjianet added backend and removed Waiting for reply Waiting for reply labels May 18, 2022

songjianet added this to the 3.0.0-beta-2 milestone May 18, 2022

qingwli mentioned this issue May 19, 2022

[Fix-10103][k8s]Fix k8s Change DataSource Error #10128

Merged

kezhenxu94 closed this as completed in #10128 May 24, 2022

This was referenced May 24, 2022

[Doc][Docker] Update Docker Doc #10226

Closed

[improve] Change Mysql Driver #10220

Merged

qingwli mentioned this issue Jun 13, 2022

[Bug][Tools] Add datasource url for mysql profile in tools application.yaml #10399

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] [deploy on k8s] SPRING_DATASOURCE_DRIVER_CLASS_NAME need add in templates/_helpers.tpl #10103

[Bug] [deploy on k8s] SPRING_DATASOURCE_DRIVER_CLASS_NAME need add in templates/_helpers.tpl #10103

zwZjut commented May 18, 2022

zwZjut commented May 18, 2022

github-actions bot commented May 18, 2022

qingwli commented May 19, 2022

qingwli commented May 19, 2022

qingwli commented May 19, 2022 •

edited

Loading

SbloodyS commented May 19, 2022

kezhenxu94 commented May 20, 2022

kezhenxu94 commented May 20, 2022 •

edited

Loading

qingwli commented May 20, 2022

kezhenxu94 commented May 20, 2022

qingwli commented May 20, 2022

qingwli commented May 20, 2022

kezhenxu94 commented May 20, 2022

kezhenxu94 commented May 20, 2022

qingwli commented May 20, 2022

kezhenxu94 commented May 20, 2022

qingwli commented May 20, 2022

SbloodyS commented May 20, 2022

kezhenxu94 commented May 20, 2022

qingwli commented May 20, 2022

qingwli commented May 20, 2022

kezhenxu94 commented May 20, 2022

qingwli commented May 24, 2022

[Bug] [deploy on k8s] SPRING_DATASOURCE_DRIVER_CLASS_NAME need add in templates/_helpers.tpl #10103

[Bug] [deploy on k8s] SPRING_DATASOURCE_DRIVER_CLASS_NAME need add in templates/_helpers.tpl #10103

Comments

zwZjut commented May 18, 2022

Search before asking

What happened

What you expected to happen

How to reproduce

Anything else

Version

Are you willing to submit PR?

Code of Conduct

zwZjut commented May 18, 2022

github-actions bot commented May 18, 2022

qingwli commented May 19, 2022

qingwli commented May 19, 2022

qingwli commented May 19, 2022 • edited Loading

SbloodyS commented May 19, 2022

kezhenxu94 commented May 20, 2022

kezhenxu94 commented May 20, 2022 • edited Loading

qingwli commented May 20, 2022

kezhenxu94 commented May 20, 2022

qingwli commented May 20, 2022

qingwli commented May 20, 2022

kezhenxu94 commented May 20, 2022

kezhenxu94 commented May 20, 2022

qingwli commented May 20, 2022

kezhenxu94 commented May 20, 2022

qingwli commented May 20, 2022

SbloodyS commented May 20, 2022

kezhenxu94 commented May 20, 2022

qingwli commented May 20, 2022

qingwli commented May 20, 2022

kezhenxu94 commented May 20, 2022

qingwli commented May 24, 2022

qingwli commented May 19, 2022 •

edited

Loading

kezhenxu94 commented May 20, 2022 •

edited

Loading