Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make run:once work for dependencies shared amongst included taskfiles #852

Closed
Place1 opened this issue Aug 31, 2022 · 2 comments · Fixed by #1655
Closed

Make run:once work for dependencies shared amongst included taskfiles #852

Place1 opened this issue Aug 31, 2022 · 2 comments · Fixed by #1655
Labels
area: fingerprinting Changes related to checksums and caching.

Comments

@Place1
Copy link

Place1 commented Aug 31, 2022

I'm currently working in a mono-repo of sorts and each of our services/libraries has a Taskfile with a "build" task.

When a service depends on a library we use "includes" and "deps" to make sure the library is built before the service is built.

We also have a root Taskfile that includes all service Taskfiles and calls their build task in a deps array.

This works really well but we've found that when a library is referenced by multiple services it will be built multiple times by the root Taskfile.

We also found that the "sources" feature doesn't work for tasks shared between included Taskfiles.

Here's the dry run from the root Taskfile showing the library getting built twice:

$ task build --dry
task: [service-a:library:build] run-build.sh 
task: [service-b:library:build] run-build.sh 
task: [service-a:build] run-build.sh 
task: [service-b:build] run-build.sh

and here's what a service Taskfile looks like

version: '3'

includes:
  web:
    taskfile: ../library/Taskfile.yml
    dir: ../library

tasks:
  build:
    run: once
    deps: [library:build]
    cmds:
      - run-build.sh
    sources:
      - src/**/*

and the library Taskfile

version: '3'

tasks:
  build:
    run: once
    cmds:
      - run-build.sh
    sources:
      - src/**/*

It seems like the include namespacing causes task to treat every task as a unique one when multiple Taskfiles include a shared task.

We also noticed that if we run a "task build" in service-a and then service-b that the .task/ checksum cache is not shared and so the library is built twice. The same applies for any order of builds, i.e. building the library then the services.

Ideally the includes form a directed acyclic graph (DAG) and task will run the dependencies in order and only once (when run:once is set), with the task's checksum cache being considered.

Is this something in scope for improvement in taskfile v3?

@TomerBin
Copy link

Hey all :)
Did anyone come up with a solution or workaround for that issue?

@pd93 pd93 added type: bug Something not working as intended. area: fingerprinting Changes related to checksums and caching. labels May 13, 2024
@pd93
Copy link
Member

pd93 commented May 13, 2024

Responding to this comment

Is #852 resolved with this?

@ReillyBrogan No, the DAG changes do not resolve this issue currently. This is because we are still merging into ast.Taskfile once the DAG reader is done reading files. This will eventually go away and resolve this issue. However, we can do something to fix it in the meantime.

See #1655 for details. Feedback on the PR would be appreciated from anyone interested in this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: fingerprinting Changes related to checksums and caching.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants