Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v2 global installation does not infer 1.x installed in repository #8309

Open
1 task done
4leite opened this issue Jun 5, 2024 · 33 comments
Open
1 task done

v2 global installation does not infer 1.x installed in repository #8309

4leite opened this issue Jun 5, 2024 · 33 comments
Labels
kind: bug Something isn't working owned-by: turborepo

Comments

@4leite
Copy link

4leite commented Jun 5, 2024

Verify canary release

  • I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

https://github.com/4leite/my-turborepo

https://github.com/4leite/my-turborepo/actions/runs/9378148823/job/25820887745

What package manager are you using / does the bug impact?

npm

What operating system are you using?

Linux

Which canary version will you have in your reproduction?

n/a

Describe the Bug

Existing github actions using:
npx -y turbo build --dry-run=json --filter=...[$1]
will pull the latest version v2.x and fails to parse the existing json due to breaking changes.

Expected Behavior

Not sure, but I'm documenting the workaround here, because it was not immediately obvious that a new release had happened or why my build stopped working.

To Reproduce

Any v1 json running:
npx -y turbo build --dry-run=json

Additional context

Workaround is to pin the version of turbo at v1
npx -y turbo@1 build --dry-run=json

@4leite 4leite added kind: bug Something isn't working needs: triage New issues get this label. Remove it after triage owned-by: turborepo labels Jun 5, 2024
@anthonyshew
Copy link
Contributor

Please provide a reproduction.

@anthonyshew anthonyshew added needs: reproduction and removed needs: triage New issues get this label. Remove it after triage labels Jun 5, 2024
@benmarch
Copy link

benmarch commented Jun 5, 2024

Was just about to report this as well. We have a Dockerfile that runs the following:

RUN npm install -g turbo
COPY . .
RUN turbo prune --scope=${APP_NAME} --docker

When that runs, I get the following error:

#18 [builder 7/7] RUN turbo prune --scope=@redacted/home --docker
#18 1.033 
#18 1.033 Attention:
#18 1.033 Turborepo now collects completely anonymous telemetry regarding usage.
#18 1.033 This information is used to shape the Turborepo roadmap and prioritize features.
#18 1.033 You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
#18 1.033 https://turbo.build/repo/docs/telemetry
#18 1.033 
#18 1.033   x could not resolve workspaces: missing packageManager field in package.json
#18 1.033   `-> missing packageManager field in package.json
#18 1.033 
#18 ERROR: executor failed running [/bin/sh -c turbo prune --scope=${APP_NAME} --docker]: exit code: 1

Looks to be thrown here: https://github.com/vercel/turbo/blob/67b2a855dbb28b0bcc4d16513f7a89b95c9c3906/crates/turborepo-repository/src/package_manager/mod.rs#L428-L431

Thanks for the workaround @4leite, worked for me.

@4leite
Copy link
Author

4leite commented Jun 5, 2024

@4leite
Copy link
Author

4leite commented Jun 5, 2024

What would have been useful is an error saying "This turbo config is for v1 and is not compatible with v2 of turbo, please update your configuration" or something similar.

@JenangMaker
Copy link

Same problem...

@JenangMaker
Copy link

Current fix is specify turbo version during docker build to 1.13.4
RUN npm install turbo@1.13.4 --global

@anthonyshew
Copy link
Contributor

Folks, can you confirm for us that you have turbo installed in the repository?

@4leite, I do see that you've done so on your reproduction, thank you. For the rest of you, have you installed turbo into your repo as well as globally?

For context, global turbo is meant to infer down (or up) to the version installed inside of your repository. The expectation is that, if you ran npm i -g turbo@latest and had an installed version of 1.x in your repository, you would be running the 1.x version (despite installing turbo@2 globally today). That would mean that a turbo.json configured for 1.x would still run as expected.

I want to make sure that we're not seeing these failures because we're running turbo@2 against a turbo.json meant for turbo@1. The bug here would be that a global installation of turbo@2 isn't inferring to turbo@1 in the repository.

@anthonyshew anthonyshew changed the title v2 release breaks existing github actions v2 global installation does not infer 1.x installed in repository Jun 5, 2024
@4leite
Copy link
Author

4leite commented Jun 5, 2024

Folks, can you confirm for us that you have turbo installed in the repository?

@4leite, I do see that you've done so on your reproduction, thank you. For the rest of you, have you installed turbo into your repo as well as globally?

For context, global turbo is meant to infer down (or up) to the version installed inside of your repository. The expectation is that, if you ran npm i -g turbo@latest and had an installed version of 1.x in your repository, you would be running the 1.x version (despite installing turbo@2 globally today). That would mean that a turbo.json configured for 1.x would still run as expected.

I want to make sure that we're not seeing these failures because we're running turbo@2 against a turbo.json meant for turbo@1. The bug here would be that a global installation of turbo@2 isn't inferring to turbo@1 in the repository.

The issue is that it's common when running the --dry-run task, particularly as a first step to decide what other steps to run, to run this globally without running npm install or similar first. I suspect the inference checks the node_modules, which is not present.

By avoiding having to install the dependencies, we significantly decrease our CI run time.

@MockusT
Copy link

MockusT commented Jun 5, 2024

Maybe updating some examples that Turbo provides could help avoid these types of issues in future when other major releases happen?

I believe quite some people follow docs to setup their apps.

Like in the Docker example - https://turbo.build/repo/docs/guides/tools/docker#example

There is a Line - RUN yarn global add turbo. It install latest version of turbo which is 2.0.1 at the time of writing. So if previously setup for turborepo was made with v1 - your build would fail.

Changing examples into something like this (self detecting version, using node):

RUN yarn global add turbo@$(node -p "require('./package.json').dependencies?.['turbo'] || require('./package.json').devDependencies['turbo']")

or this (leave for users to write specific version):

RUN yarn global add turbo@<turborepo-version-for-the-app>

(or even some other, better solutions) could prevent build failures when other major releases happen.

@anthonyshew
Copy link
Contributor

@MockusT I agree - but that's different from the bug we're seeing here. We're trying to make sure that global 2.x is handling repo-inference correctly.

@buzzie-bee
Copy link

I had this issue today but it was slightly different to how it's been described here.

My docker builds were failing after running prune at the build step.

When I went into the failing layers and ran npx turbo --version it reported that it was still using turbo v1.13.3. However the turbo.json that was output from the prune step was missing the "pipeline" key and throwing the following error:

Step 18/26 : RUN npx turbo build --filter=backend...
 ---> Running in ed7a6fee8ac1

  x missing tasks in project

Error:   x could not find task `build` in project

This makes me think that even though turbo v1.13.3 was reported as being used (the same version as is in my package.json), the prune step was using the latest "$schema": "https://turbo.build/schema.json", schema which is missing the "pipeline" key from RootSchema and was stripping it out of the pruned turbo.json.

I've since updated to turbo v2.0.0 and the issue is fixed, but it took a while to debug and may be confusing to a lot of turbo users.

@NicholasLYang
Copy link
Contributor

Hey, yeah, there will be some awkwardness if global turbo is 2.0 while local turbo is 1.0. Specifically, we now require the packageManager field in package.json in 2.0, so global turbo's inference will fail if that's no packageManager field, even if local turbo still accepts it. You can run the add-package-manager codemod by running npx @turbo/codemod to solve this. There's likely some other small incompatibilities. We'll try to document this as best we can, but in the meanwhile, we recommend keeping the local turbo the same major version as your global turbo.

@anthonyshew
Copy link
Contributor

@4leite I made a mistake earlier. The reproduction you've provided can't infer down to a 1.x version since it was never installed, so it makes sense that Turborepo 2.0 can't interpret a configuration from 1.x.

You can either set a version for the npx turbo invocation (e.g. npx turbo@^1 ...) or follow the upgrade guide to get upgraded.

@anthonyshew
Copy link
Contributor

@buzzie-bee The $schema field is for in-editor help only. It doesn't affect runtime. The issue you're likely seeing there is inadvertently using a global turbo prune prior to package installation, which would use 2.0. However, after installing packages in the Docker image, 1.13 would be available and used.

@chris-olszewski
Copy link
Member

@buzzie-bee

When I went into the failing layers and ran npx turbo --version it reported that it was still using turbo v1.13.3.

Was this post-prune and inside the output directory? Was the Step 18/26 : RUN npx turbo build --filter=backend... run after a npm ci in the output directory?

@buzzie-bee
Copy link

buzzie-bee commented Jun 5, 2024

@chris-olszewski

Was this post-prune and inside the output directory?

Yes this is post prune.

I have a build step which runs RUN npx turbo prune backend --docker, then in another step I copy the out dir, then npm ci, then copy in source files, then the npx turbo build --filter=backend....

What @anthonyshew suggested is likely what was going on. When I ran it the first time it used v2.0.0 to do the prune, then after installing it used the locally installed v1.13.3 version which makes sense.

Maybe for the future, the docs can suggest to use a pinned version in the examples so a potential v3.0.0 doesn't cause the same issues if the schema changes? I had followed this page I think when I first set it up: https://turbo.build/repo/docs/guides/tools/docker

Thanks both for getting back to me so quickly :)

@4leite
Copy link
Author

4leite commented Jun 5, 2024

@4leite I made a mistake earlier. The reproduction you've provided can't infer down to a 1.x version since it was never installed, so it makes sense that Turborepo 2.0 can't interpret a configuration from 1.x.

You can either set a version for the npx turbo invocation (e.g. npx turbo@^1 ...) or follow the upgrade guide to get upgraded.

@anthonyshew Yes, pinning the version is literally what I did as mentioned on the ticket as my workaround.

Sorry but "just upgrade" is not good advice for a large project on day 0 of a new release.

I'm not asking for you to fix the inference - I agree there's not likely a good solution for this.

I'm asking for a better error message when your release breaks existing behavior. "JSON parsing error" gives no indication to the user that what has actually happened is that an unknown new release has broken support for the current features.

Usual practice is to deprecate with a warning first and remove in subsequent releases.

@dfkadyr-old
Copy link

dfkadyr-old commented Jun 6, 2024

@anthonyshew Hi.
The "Ignored Build Step" logic hasn't worked for two days now when the "Only build Turborepo app if there are changes" option is selected.

I get the following error:

image

Locally, I'm using "turbo": "1.13.2" for the project.

Any ideas on how to fix this?

@anthonyshew
Copy link
Contributor

You can use a custom ignore step and set it to npx turbo-ignore@^1 as long as you're using v1 configuration. We're working on having a better error message here.

@anthonyshew
Copy link
Contributor

anthonyshew commented Jun 7, 2024

Hey, folks, want to update here. We're aware that this upgrade process wasn't smooth enough and I want to update with some fixes we've put in place.

Code updates

  • We now log a warning when no local turbo is found on an invocation of global turbo to help you know when you are out of sync (PR)
  • turbo-ignore now infers the correct version for your project in as many cases as we can reasonably do so (PR)

Documentation updates

The old documentation wasn't good enough to keep you safe during this transition. We made some changes in the initial documentation release and several more over the past couple days.

We can't change the difficulties that we created for you in the near past. Instead, we can do our best for you in the future. From working with folks around the ecosystem and here in this thread, we're finding these changes have helped significantly already.

We're also still investigating one larger change for global turbo that would keep you always on the version meant for your repo, no intervention from you required (similar to Corepack for package managers). I'm going to leave this open until we see what comes of that work.

@mpereira
Copy link

mpereira commented Jun 9, 2024

After upgrading to Turbo v2, my Vercel builds also started failing:

package.json:

{
  "name": "<redacted>",
  "version": "0.0.0",
  "workspaces": [
    "apps/*",
    "packages/*"
  ],
  "private": true,
  "packageManager": "npm@10.8.0",
  "devDependencies": {
    "npm-check-updates": "16.14.20",
    "turbo": "2.0.3"
  },
  "scripts": {
    "upgrade-dependencies": "ncu -u"
  }
}

turbo.json:

{
  "$schema": "https://turbo.build/schema.json",
  "tasks": {
    "analyze-dependencies": {},
    "build": {
      "outputs": [
        ".next/**",
        "!.next/cache/**"
      ],
      "dependsOn": [
        "^build"
      ]
    },
    "check-dependencies": {},
    "dev": {
      "cache": false,
      "persistent": true
    },
    "format-files": {},
    "lint": {},
    "start": {},
    "test": {
      "inputs": [
        "app/**/*.tsx",
        "app/**/*.ts",
        "test/**/*.ts",
        "test/**/*.tsx"
      ]
    },
    "//#upgrade-dependencies": {},
    "upgrade-dependencies": {}
  }
}
[18:27:14.393] Running build in Washington, D.C., USA (East) – iad1
[18:27:14.552] Cloning github.com/<redacted> (Branch: <redacted>, Commit: <redacted>)
[18:27:15.907] Cloning completed: 1.354s
[18:27:21.008] Restored build cache
[18:27:21.169] Running "vercel build"
[18:27:21.730] Vercel CLI 34.2.6
[18:27:22.153] Installing dependencies...
[18:27:36.608] 
[18:27:36.609] added 132 packages, removed 90 packages, and changed 139 packages in 14s
[18:27:36.609] 
[18:27:36.609] 227 packages are looking for funding
[18:27:36.609]   run `npm fund` for details
[18:27:36.637] Detected Next.js version: 14.2.3
[18:27:36.638] Running "turbo run build"
[18:27:36.738] 
[18:27:36.739] Attention:
[18:27:36.740] Turborepo now collects completely anonymous telemetry regarding usage.
[18:27:36.740] This information is used to shape the Turborepo roadmap and prioritize features.
[18:27:36.740] You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
[18:27:36.740] https://turbo.build/repo/docs/telemetry
[18:27:36.741] 
[18:27:36.743] turbo_json_parse_error
[18:27:36.743] 
[18:27:36.743]   x failed to parse turbo json
[18:27:36.743] 
[18:27:36.744] Error: turbo_json_parse_error
[18:27:36.744] 
[18:27:36.744]   x Found an unknown key `tasks`.
[18:27:36.744]    ,-[2:1]
[18:27:36.744]  2 |   "$schema": "https://turbo.build/schema.json",
[18:27:36.744]  3 |   "tasks": {
[18:27:36.744]    :   ^^^^^^^
[18:27:36.744]  4 |     "analyze-dependencies": {},
[18:27:36.744]    `----
[18:27:36.744] 
[18:27:36.752] Error: Command "turbo run build" exited with 1
[18:27:37.106] }

@brunofin
Copy link

this is happening on my repo. Preview builds are working but production builds are throwing this error.

@cSarcasme
Copy link

Same issue in development work perfectly but when i push on vercel impossible to deploy same issue...

Error: turbo_json_parse_error
  x Found an unknown key `tasks`.
   ,-[4:1]
 4 |     "globalEnv": ["PORT"],
 5 |     "tasks": {
   :     ^^^^^^^
 6 |         "build": {
   `----
Error: Command "turbo run build" exited with 1

Look for a solution impossible to find one for the moment ty.

@brunofin
Copy link

Same issue in development work perfectly but when i push on vercel impossible to deploy same issue...

Error: turbo_json_parse_error
  x Found an unknown key `tasks`.
   ,-[4:1]
 4 |     "globalEnv": ["PORT"],
 5 |     "tasks": {
   :     ^^^^^^^
 6 |         "build": {
   `----
Error: Command "turbo run build" exited with 1

Look for a solution impossible to find one for the moment ty.

I've opened a discussion here: https://github.com/orgs/vercel/discussions/7231#discussioncomment-9809507
Feel free to comment and upvote for visibility.

@brunofin
Copy link

I just solved the problem by moving my dependency to turbo from devDependencies to dependencies:

{
  "name": "<redacted>-monorepo",
  "private": true,
  "scripts": {
    "build": "turbo build",
    "dev": "turbo dev",
    "lint": "turbo lint",
    "format": "prettier --write \"**/*.{ts,tsx,md}\""
  },
  "devDependencies": {
    "@repo/eslint-config": "*",
    "@repo/typescript-config": "*",
    "prettier": "^3.2.5"
  },
  "dependencies": {
    "turbo": "^2.0.3"
  },
  "engines": {
    "node": "20",
    "yarn": "1.22"
  },
  "packageManager": "yarn@1.22.19",
  "workspaces": [
    "apps/*",
    "packages/@<redacted1>/*",
    "packages/@<redacted2>/*"
  ]
}

@cSarcasme
Copy link

i just try but change nothing when deploy on vercel always same error

Error: turbo_json_parse_error
  x Found an unknown key `tasks`.
   ,-[4:1]
 4 |     "globalEnv": ["PORT"],
 5 |     "tasks": {
   :     ^^^^^^^
 6 |         "build": {
   `----
Error: Command "cd ../.. && turbo run build --filter={apps/frontend}..." exited with 1

Also i use npm:

This is my root package.json file

{
	"private": true,
	"scripts": {
		"build": "turbo run build",
		"clean": "turbo run clean",
		"dev": "turbo run dev",
		"format": "prettier --write \"**/*.{ts,tsx,md}\"",
		"lint": "turbo run lint",
		"test": "turbo run test",
		"typecheck": "turbo run typecheck"
	},
	"depedencies": {
		"turbo": "^2.0.4"
	},
	"devDependencies": {
		"prettier": "^3.2.5"
	},
	"engines": {
		"node": ">=18"
	},
	"name": "kitchen-sink",
	"packageManager": "npm@10.1.0",
	"workspaces": [
		"apps/*",
		"packages/*"
	]
}

My root turbo.json

{
	"$schema": "https://turbo.build/schema.json",
	"globalDependencies": ["**/.env.*local"],
	"globalEnv": ["PORT"],
	"tasks": {
		"build": {
			"dependsOn": ["^build"]
		},
		"test": {
			"outputs": ["coverage/**"],
			"dependsOn": []
		},
		"lint": {
			"dependsOn": ["^build"]
		},
		"typecheck": {
			"dependsOn": ["^build"]
		},
		"dev": {
			"dependsOn": ["^build"],
			"cache": false,
			"persistent": true
		},
		"clean": {
			"cache": false
		}
	}
}

My turbo.json of my backend folder whos is in apps/backend who work with express and pug.


{
	"extends": ["//"],
	"tasks": {
		"build": {
			"outputs": ["dist/**"]
		}
	}
}

My turbo.json file in my folder frontend in apps/frontend who work with nextjs.

{
	"extends": ["//"],
	"tasks": {
		"build": {
			"outputs": [".next/**", "!.next/cache/**"]
		}
	}
}

When i run the command turbo run dev work perfectly locally but impossible to makes work on deploy on vercel.

If anyone has the solution i am really need it.

@brunofin
Copy link

@cSarcasme what I also did was to remove all node_modules and yarn.lock files, run yarn install again only on the monorepo root folder to generate a new lock file, and push it.

find . -name "node_modules" -type d -prune -exec rm -rf '{}' +
find . -name "yarn.lock" -type f -prune -exec rm -rf '{}' +

@brunofin
Copy link

Also if possible, you could try starting your turbo repo from scratch, because only 2 days ago all templates were updated to fix a few issues including the one I ran into: #8192

That is somewhat extreme but could help.

@anthonyshew
Copy link
Contributor

To update, we're aware of this class of error on Vercel and working towards releasing the fix.

@mpereira
Copy link

@anthonyshew thanks for the ack.

@omarish
Copy link

omarish commented Jun 19, 2024

Running into this as well with an upgrade from 1.x.x to 2.0.4.

@anthonyshew
Copy link
Contributor

anthonyshew commented Jun 20, 2024

Folks, we believe we fixed this error path on Vercel. Could I ask those of you who were running into this before to try a re-deploy?

cc @omarish @brunofin @cSarcasme @mpereira 🙏

@mpereira
Copy link

mpereira commented Jun 20, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: bug Something isn't working owned-by: turborepo
Projects
None yet
Development

No branches or pull requests