feat: Allow to continue template execution using "ignoreError" property

[BenjaminDecreusefond]

Hi @alfespa17 !

I'd like to propose this small change and add a new field to the Terrakube template. I change the Flow template in the API and added it to the Executor job.

However, I'd like your inputs about it, I think I'm missing the part where the data is transmitted from the API that parse the yaml to the Executor that use the content of the Yaml to run the Job. I tried to find it but I couldn't find the connection between the 2 components unfortunately. My guess is that the API set the job in the queue and the executor will pick'em for execute. I'm really not sure that the field ingore_error will be available in the Executor context. Could you explain me please ? :)

Regards

[alfespa17]

This looks better, let me check the logic that you have changed, and I will let you know if it requires any additional change

[alfespa17]

This is the part of the code that is parsing the YAML file

The job table has a field called "tcl" that has the base64 encoded yaml that will be executed when the job is running.

terrakube/api/src/main/java/org/terrakube/api/rs/job/Job.java

Line 71 in a069946

private String tcl;

This is the service that parse the yaml into a java class:

terrakube/api/src/main/java/org/terrakube/api/plugin/scheduler/job/tcl/TclService.java

Line 82 in a069946

FlowConfig temp = yaml.load(new String(Base64.getDecoder().decode(tcl)));

By the way TCL means "terrakube configuration language"

When the job is running it is checking the pending steps here

terrakube/api/src/main/java/org/terrakube/api/plugin/scheduler/ScheduleJob.java

Line 159 in a069946

Optional<Flow> flow = Optional.ofNullable(tclService.getNextFlow(job));

Parsing the pending steps from here

terrakube/api/src/main/java/org/terrakube/api/plugin/scheduler/job/tcl/TclService.java

Line 111 in a069946

public Flow getNextFlow(Job job) {

And it validates the pending job parts here

terrakube/api/src/main/java/org/terrakube/api/plugin/scheduler/ScheduleJob.java

Line 159 in a069946

Optional<Flow> flow = Optional.ofNullable(tclService.getNextFlow(job));

[BenjaminDecreusefond]

I do follow you until there
Optional<Flow> flow = Optional.ofNullable(tclService.getNextFlow(job)); however I have trouble to understand why this on the plan seem to do nothing ? I don't understand what happens when we are using the terraformPlan type ?

[alfespa17]

I do follow you until there Optional<Flow> flow = Optional.ofNullable(tclService.getNextFlow(job)); however I have trouble to understand why this on the plan seem to do nothing ? I don't understand what happens when we are using the terraformPlan type ?

All of those case are executing the same code:

                case terraformPlanDestroy:
                case terraformPlan:
                case terraformApply:
                case terraformDestroy:
                case customScripts:
                    if (executorService.execute(job, stepId, flow.get()) != null)
                        log.info("Executing Job {} Step Id {}", job.getId(), stepId);
                    else {
                        log.error("Error when sending context to executor marking job {} as failed, step count {}", job.getId(), job.getStep().size());
                        job.setStatus(JobStatus.failed);
                        jobRepository.save(job);
                        Step step = stepRepository.getReferenceById(UUID.fromString(stepId));
                        step.setName("Error sending to executor, check logs");
                        stepRepository.save(step);
                    }
                    break;

That is a way to reuse the case with the same logic that begins in if (executorService.execute(job, stepId, flow.get()) != null)

[BenjaminDecreusefond]

Okayy thank you @alfespa17 that's a Java trick ! I didn't know it !

I added the value to the ExecutorContext which should then allow the Executor code to retrieve the value, am I right ?

[alfespa17]

Okayy thank you @alfespa17 that's a Java trick ! I didn't know it !

I added the value to the ExecutorContext which should then allow the Executor code to retrieve the value, am I right ?

I think your change should work just one small thing can you rename ignore_error to ignoreError so we use the same variable name standard?

[BenjaminDecreusefond]

Sure! Done ! :) @alfespa17

[alfespa17]

I tested with this and it is working, it is ignoring the terraform plan error and it is running the script

flow:
- type: "terraformPlan"
  name: "Plan"
  step: 100
  ignoreError: true
  commands:
  - runtime: "BASH"
    priority: 100
    after: true
    script: |
      echo "Running after error"

[BenjaminDecreusefond]

Hi @alfespa17 !

Would it be possible to have a release please ? :)

thanks ! :)

[alfespa17]

You can check 2.24.0-beta.6 @BenjaminDecreusefond