[Feature] Add project name/workflow definition name/instance id/task name to the built-in variables #14099

haibingtown · 2023-05-14T08:00:51Z

Purpose of the pull request

Brief change log

Add project name/workflow definition name/instance id/task name to the built-in variables

Verify this pull request

This pull request is code cleanup without any test coverage.

zhongjiajie

LGTM overall, just lift some comments. BTW can you add some docs in file https://github.com/apache/dolphinscheduler/blob/d46e4659d917e1994bc9045c57fe3b865a308352/docs/docs/zh/guide/parameter/built-in.md ?

...ler-service/src/main/java/org/apache/dolphinscheduler/service/expand/CuringGlobalParams.java

...eduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/TaskConstants.java

haibingtown · 2023-05-15T15:25:28Z

@zhongjiajie The comments are spot on, they have been resolved

calvinjiang

Please take a look at those tips that I offer.

...ler-service/src/main/java/org/apache/dolphinscheduler/service/expand/CuringGlobalParams.java

docs/docs/en/guide/parameter/built-in.md

zhuangchong · 2023-05-16T06:48:56Z

Please execute the spotless command mvn spotless:apply in the local project to format the document style. @haibingtown

codecov-commenter · 2023-05-17T01:24:12Z

Codecov Report

Merging #14099 (79707ab) into dev (8315279) will decrease coverage by 0.08%.
The diff coverage is 44.64%.

❗ Current head 79707ab differs from pull request most recent head 59490f3. Consider uploading reports for the commit 59490f3 to get more accurate results

@@             Coverage Diff              @@
##                dev   #14099      +/-   ##
============================================
- Coverage     38.41%   38.33%   -0.08%     
- Complexity     4461     4470       +9     
============================================
  Files          1223     1229       +6     
  Lines         42549    42887     +338     
  Branches       4716     4762      +46     
============================================
+ Hits          16345    16442      +97     
- Misses        24403    24622     +219     
- Partials       1801     1823      +22

Impacted Files	Coverage Δ
...nscheduler/api/controller/SchedulerController.java	`82.60% <ø> (ø)`
...heduler/api/service/impl/SchedulerServiceImpl.java	`30.13% <0.00%> (-0.09%)`	⬇️
...cheduler/common/log/remote/S3RemoteLogHandler.java	`0.00% <0.00%> (ø)`
.../apache/dolphinscheduler/common/utils/OSUtils.java	`29.55% <0.00%> (-0.38%)`	⬇️
.../master/processor/MasterTaskDispatchProcessor.java	`0.00% <0.00%> (ø)`
...rver/master/processor/MasterTaskKillProcessor.java	`0.00% <0.00%> (ø)`
...ver/master/processor/MasterTaskPauseProcessor.java	`0.00% <0.00%> (ø)`
...asterDelayTaskExecuteRunnableDelayQueueLooper.java	`0.00% <0.00%> (ø)`
.../server/master/runner/WorkflowExecuteRunnable.java	`10.38% <0.00%> (+0.04%)`	⬆️
...unner/execute/AsyncMasterTaskDelayQueueLooper.java	`0.00% <0.00%> (ø)`
... and 22 more

... and 7 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

zhuangchong · 2023-05-18T01:24:06Z

@haibingtown
At present, The CI reports an error: the front-end module build is abnormal, but this pr does not change the front-end code. Can you merge the latest code of the dev branch to try to see if this problem is solved?

caishunfeng

LGTM 👍

haibingtown · 2023-05-18T03:13:47Z

@zhuangchong Still having issues with CI, what do I need to do

zhuangchong · 2023-05-18T10:53:46Z

@zhuangchong Still having issues with CI, what do I need to do

@haibingtown The dev branch has just solved this problem, can you merge it once?

haibingtown · 2023-05-18T12:27:01Z

@zhuangchong ok, done

haibingtown · 2023-05-19T05:34:35Z

@zhuangchong @calvinjiang @caishunfeng i guess it seems that this CI still needs a brother to help

SbloodyS

+1

SbloodyS · 2023-05-19T08:23:05Z

@SbloodyS I don't think this failure has anything to do with my code, or can you give me some pointers on where to look for this problem. At the same time, it seems that the coverage rate has not passed, who can help to see。

The workflow instance execution failed. You can download this video to simulate the operation locally.
https://github.com/apache/dolphinscheduler/suites/12985499794/artifacts/704246970

haibingtown · 2023-05-19T11:58:50Z

@SbloodyS @zhuangchong @calvinjiang I have cherry-pick my commits, can you guys trigger CI again

…name to the built-in variables

zhongjiajie · 2023-05-22T06:41:33Z

@SbloodyS I don't think this failure has anything to do with my code, or can you give me some pointers on where to look for this problem. At the same time, it seems that the coverage rate has not passed, who can help to see。

The workflow instance execution failed. You can download this video to simulate the operation locally. https://github.com/apache/dolphinscheduler/suites/12985499794/artifacts/704246970

FYI, we only have one day for our docker images, so if you may sure your code is correct you can re-run all tests to avoid docker image expired

zhongjiajie · 2023-05-23T02:01:36Z

approval the ci run

zhongjiajie · 2023-05-24T01:45:40Z

restart the failed ci

docs/docs/zh/guide/parameter/built-in.md

docs/docs/en/guide/parameter/built-in.md

calvinjiang · 2023-05-24T10:19:50Z

@SbloodyS @zhuangchong @calvinjiang I have cherry-pick my commits, can you guys trigger CI again
@haibingtown
I just restarted the E2E test. But it seems like that there is an error with running workflow. You can check the class 'WorkflowE2ETest' and the test case 'testRunWorkflow'. You'll find that the whole test class includes three test cases in order. Before running the test case 'testRunWorkflow' the test case 'testCreateWorkflow' and 'testCreateSubWorkflow' needs to be ran. As you can see, the two test cases use some parameters that might cause the error. You should simulate these settings in your workflow and see if there is the same error.

The error log is as follows:

…built-in variables

zhuangchong · 2023-05-25T02:20:14Z

@haibingtown
I fixed the E2E exception, the cause of the exception is that timeZone is empty when manually triggering the workflow.

dolphinscheduler/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/expand/CuringGlobalParams.java

Lines 217 to 219 in 29e31d8

    
           private Map<String, String> setBuiltInParamsMap(@NonNull TaskInstance taskInstance, 
        
                                                           @NonNull String timeZone) { 
        
               CommandType commandType = taskInstance.getProcessInstance().getCmdTypeIfComplement();

zhuangchong · 2023-05-25T02:23:30Z

@haibingtown

Thank you for your valuable contribution. Your PR is excellent! I would like to establish a deeper communication with you. You can reach me via email or add me on WeChat (KerwinZhuang). When contacting me, please provide your identity for reference. If you encounter any difficulties with DolphinScheduler, I am more than happy to assist you and help you become familiar with the platform.

sonarcloud · 2023-05-25T02:48:35Z

SonarCloud Quality Gate failed.

0 Bugs
0 Vulnerabilities
0 Security Hotspots
15 Code Smells

32.0% Coverage
0.0% Duplication

calvinjiang

LGTM

yihleego · 2023-07-31T07:16:04Z

LGTM
I found a typo: ${ssystem.task.instance.id}, plz fix it.

https://github.com/apache/dolphinscheduler/blob/150d9e04b0675e3d4bee6000a742a0e54846472e/docs/docs/en/guide/parameter/built-in.md?plain=1#L11C40-L11C47

https://github.com/apache/dolphinscheduler/blob/150d9e04b0675e3d4bee6000a742a0e54846472e/docs/docs/zh/guide/parameter/built-in.md?plain=1#L11C40-L11C47

Gallardot · 2023-07-31T08:34:56Z

LGTM I found a typo: ${ssystem.task.instance.id}, plz fix it.

https://github.com/apache/dolphinscheduler/blob/150d9e04b0675e3d4bee6000a742a0e54846472e/docs/docs/en/guide/parameter/built-in.md?plain=1#L11C40-L11C47

https://github.com/apache/dolphinscheduler/blob/150d9e04b0675e3d4bee6000a742a0e54846472e/docs/docs/zh/guide/parameter/built-in.md?plain=1#L11C40-L11C47

@yihleego Thanks, I have fixed it. PTAL. #14667

…name to the built-in variables apache#14099

haibingtown requested review from caishunfeng, SbloodyS and zhuangchong as code owners May 14, 2023 08:00

github-actions bot added the backend label May 14, 2023

zhongjiajie reviewed May 15, 2023

View reviewed changes

zhongjiajie added the miss:docs missing documents in PR label May 15, 2023

SbloodyS added the first time contributor First-time contributor label May 15, 2023

SbloodyS assigned haibingtown May 15, 2023

SbloodyS added the feature new feature label May 15, 2023

haibingtown force-pushed the Feature-14007 branch from 8ea14ce to 2bd88ae Compare May 15, 2023 15:13

haibingtown requested a review from EricGao888 as a code owner May 15, 2023 15:13

github-actions bot added the document label May 15, 2023

haibingtown force-pushed the Feature-14007 branch 2 times, most recently from 9cd3118 to 89c94a1 Compare May 15, 2023 15:18

haibingtown mentioned this pull request May 15, 2023

[Feature] Add project id/workflow definition id/instance id to the built-in variables #14007

Closed

3 tasks

calvinjiang added this to the 3.2.0 milestone May 16, 2023

calvinjiang requested changes May 16, 2023

View reviewed changes

...ler-service/src/main/java/org/apache/dolphinscheduler/service/expand/CuringGlobalParams.java Outdated Show resolved Hide resolved

docs/docs/en/guide/parameter/built-in.md Outdated Show resolved Hide resolved

haibingtown force-pushed the Feature-14007 branch from 258a3a0 to 39476da Compare May 16, 2023 15:28

zhuangchong added the 3.2.0 for 3.2.0 version label May 17, 2023

haibingtown requested a review from calvinjiang May 18, 2023 03:05

caishunfeng previously approved these changes May 18, 2023

View reviewed changes

SbloodyS previously approved these changes May 19, 2023

View reviewed changes

haibingtown force-pushed the Feature-14007 branch from f03965f to 47a8a4a Compare May 19, 2023 11:14

haibingtown dismissed stale reviews from SbloodyS and caishunfeng via f86b75d May 20, 2023 08:51

haibingtown force-pushed the Feature-14007 branch from 47a8a4a to f86b75d Compare May 20, 2023 08:51

haibingtown added 3 commits May 20, 2023 23:12

[Feature] Add project name/workflow definition name/instance id/task …

5f648b4

…name to the built-in variables

[Feature-14007] fix some typo

ae3c8c1

[Feature-14007][UT] add ut

2964710

haibingtown force-pushed the Feature-14007 branch from f86b75d to 460318b Compare May 20, 2023 15:14

Modify the comment to built-in

e364aa5

haibingtown force-pushed the Feature-14007 branch from 460318b to e364aa5 Compare May 22, 2023 13:03

zhongjiajie reviewed May 24, 2023

View reviewed changes

docs/docs/zh/guide/parameter/built-in.md Outdated Show resolved Hide resolved

docs/docs/en/guide/parameter/built-in.md Show resolved Hide resolved

calvinjiang previously approved these changes May 24, 2023

View reviewed changes

[Feature] Add project code/workflow definition code/task code to the …

29e31d8

…built-in variables

haibingtown dismissed calvinjiang’s stale review via 29e31d8 May 24, 2023 13:24

fix ci error

59490f3

calvinjiang approved these changes May 25, 2023

View reviewed changes

calvinjiang merged commit af7da06 into apache:dev May 25, 2023

Gallardot mentioned this pull request Jul 31, 2023

[Doc] fix built-in typo #14667

Merged

ly109974 pushed a commit to ly109974/dolphinscheduler that referenced this pull request Apr 26, 2024

[Feature] Add project name/workflow definition name/instance id/task …

8a90c1d

…name to the built-in variables apache#14099

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Add project name/workflow definition name/instance id/task name to the built-in variables #14099

[Feature] Add project name/workflow definition name/instance id/task name to the built-in variables #14099

haibingtown commented May 14, 2023 •

edited by SbloodyS

Loading

zhongjiajie left a comment •

edited

Loading

haibingtown commented May 15, 2023

calvinjiang left a comment

zhuangchong commented May 16, 2023

codecov-commenter commented May 17, 2023 •

edited

Loading

zhuangchong commented May 18, 2023

caishunfeng left a comment

haibingtown commented May 18, 2023

zhuangchong commented May 18, 2023

haibingtown commented May 18, 2023

haibingtown commented May 19, 2023

SbloodyS left a comment

SbloodyS commented May 19, 2023

haibingtown commented May 19, 2023

zhongjiajie commented May 22, 2023

zhongjiajie commented May 23, 2023

zhongjiajie commented May 24, 2023

calvinjiang commented May 24, 2023 •

edited

Loading

zhuangchong commented May 25, 2023

zhuangchong commented May 25, 2023

sonarcloud bot commented May 25, 2023

calvinjiang left a comment

yihleego commented Jul 31, 2023

Gallardot commented Jul 31, 2023

[Feature] Add project name/workflow definition name/instance id/task name to the built-in variables #14099

[Feature] Add project name/workflow definition name/instance id/task name to the built-in variables #14099

Conversation

haibingtown commented May 14, 2023 • edited by SbloodyS Loading

Purpose of the pull request

Brief change log

Verify this pull request

zhongjiajie left a comment • edited Loading

Choose a reason for hiding this comment

haibingtown commented May 15, 2023

calvinjiang left a comment

Choose a reason for hiding this comment

zhuangchong commented May 16, 2023

codecov-commenter commented May 17, 2023 • edited Loading

Codecov Report

zhuangchong commented May 18, 2023

caishunfeng left a comment

Choose a reason for hiding this comment

haibingtown commented May 18, 2023

zhuangchong commented May 18, 2023

haibingtown commented May 18, 2023

haibingtown commented May 19, 2023

SbloodyS left a comment

Choose a reason for hiding this comment

SbloodyS commented May 19, 2023

haibingtown commented May 19, 2023

zhongjiajie commented May 22, 2023

zhongjiajie commented May 23, 2023

zhongjiajie commented May 24, 2023

calvinjiang commented May 24, 2023 • edited Loading

zhuangchong commented May 25, 2023

zhuangchong commented May 25, 2023

sonarcloud bot commented May 25, 2023

calvinjiang left a comment

Choose a reason for hiding this comment

yihleego commented Jul 31, 2023

Gallardot commented Jul 31, 2023

haibingtown commented May 14, 2023 •

edited by SbloodyS

Loading

zhongjiajie left a comment •

edited

Loading

codecov-commenter commented May 17, 2023 •

edited

Loading

calvinjiang commented May 24, 2023 •

edited

Loading