-
-
Notifications
You must be signed in to change notification settings - Fork 196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make it possible tns
to use Yarn as the package manager
#2737
Comments
No it is not possible at the moment. |
Wait how impossible? This should be a feature request. |
Reopen, title update and label |
tns
use Yarn as the package manager?tns
to use Yarn as the package manager
Any word on this? Any milestones so people can help by contributing ? |
Hey @ibrahimab ,
Of course, this is just an idea for implementation and any other suggestions and implementations are welcome. |
Hi @rosen-vladimirov thank you for your response. I want to take this up. How would this work? I want to setup milestones, tasks, issues etc. Do I do this on this repository or link to my own fork? |
Hey @ibrahimab , Regarding the tasks and issues, I would suggest to add them here, as comments, so anyone who wants to help, can take a look at the current status of the task. The other option is to create the issues in your own fork and link them in the comment here. |
Hey @ibrahimab , |
@dcarrot2 and myself are looking into implementing this. We a little frustrated on how npm does not uses package-lock.json to lock dependencies. We are also looking into take advantage of yarn's improved performance. |
@rosen-vladimirov A few questions we have.
Thanks! |
Hey guys, I'm happy you're gonna try implementing this feature. I'll be glad to help you and review any code or idea you have.
|
Yes, we're underway implementing it, but @mflor35 and I are traveling this week. We've hammered most of yarn's implementation except install. We noticed there isn't a dry run flag that we can pass to yarn, and yarn streams it's json format as the installation occurs--for all events in the routine. We're roughly thinking of the following approach to address installation and fulfill the adopted interface.
Thoughts @rosen-vladimirov? |
Hey @dcarrot2 , |
@dcarrot2 Are you implementing this in a way that would enable yarn's workspaces feature? That would allow NativeScript apps to be used in monorepos. Not sure if this requires other changes to NativeScript though |
Thanks for the guidance @rosen-vladimirov! Will take a look at pacote--looks like that should simply things greatly. @BenLorantfy Will scope that out once we've hammered out this feature request. |
The settings for selected package manager is persisted in Acceptance criteria:
|
I built the master branch locally and tried it out. Package installation seems to be working as expected. But when I run As I'm using nativescript for the first time, I can only guess from the console output that there is some package resolution defect in I setup the project using vue-cli following quick start. One thing might worth noting is that I created the project inside a pre-existing lerna monorepo and common node dependencies are hoisted to the root
|
After digging into the source code, I found that PluginService only searches in the |
I see you're using yarn and wonder if the problem is reproducible when |
@Fatme It is yarn-specific. Since npm installs all dependencies just under the project directory (as it doesn't know anything about yarn workspace and monorepo), the Yarn, however, collects dependencies to the root I suppose these functions (there may be some others) of the ps: There was a markdown formatting error in my previous comment which causes |
Technically speaking, module resolution algorithm of nodejs |
Changing getPackageJsonFilePathForModule(moduleName, projectDir) {
return require.resolve(path.join(moduleName, 'package.json'), { paths: [projectDir] })
} Still working on it. |
After solving the errors one by one, I'm finally at a stage where a fresh new project created by Although I'd like to make a PR, I cannot make the test pass for now. Because I typed these tiral-and-error-codes by solving the errors one after another, instead of getting a full picture of the codebase first and solving it systematically, I do not really have the overall understanding to be able to modify the test to make it pass. And it's also very likely to have some corner cases omitted. So, I'm attaching the diff here and hoping that someone who is more capable to make the PR. @Fatme Would you mind to take a glance ? diff
diff --git a/lib/services/plugins-service.ts b/lib/services/plugins-service.ts
index f18a9e8..e829e4b 100644
--- a/lib/services/plugins-service.ts
+++ b/lib/services/plugins-service.ts
@@ -2,6 +2,7 @@ import * as path from "path";
import * as shelljs from "shelljs";
import * as semver from "semver";
import * as constants from "../constants";
+const walkup = require("node-walkup");
export class PluginsService implements IPluginsService {
private static INSTALL_COMMAND_NAME = "install";
@@ -164,18 +165,32 @@ export class PluginsService implements IPluginsService {
}
}
+ private async getAllInstalledDependencies (projectData: IProjectData) {
+ // walk through ancestor directories and collect 'node_modules' paths
+ const nodeModulesPaths = await new Promise<string[]>((resolve, reject) => {
+ walkup('node_modules', { cwd: projectData.projectDir }, (err: any, matches: { dir: string, files: string[] }[]) =>
+ err ? reject(err) : resolve(matches.map(match => path.join(match.dir, 'node_modules')))
+ );
+ });
+
+ // get dependencies in one specific node_modules directory
+ const getDepsIn = (dir: string): (string | string[])[] => {
+ return this.$fs.readDirectory(dir)
+ .map(dep => {
+ if (!dep.startsWith('@')) { return dep; } // return directly if it's normal module
+ const scopedDir = path.join(dir, dep); // scoped module
+ return this.$fs.readDirectory(scopedDir).map(scopedDep => `${dep}/${scopedDep}`);
+ });
+ };
+ return _(nodeModulesPaths) // [ '/path/to/project/node_modules', '/path/to/project/packages/sub-project/node_modules' ]
+ .map(getDepsIn) // [ ['moduleA', 'moduleB'], ['moduleC', ['@d/moduleE', '@d/moduleF'] ], ['moduleB'] ]
+ .flattenDeep<string>() // [ 'moduleA', 'moduleB', 'moduleC', '@d/moduleE', '@d/moduleF', 'moduleB' ]
+ .uniq() // [ 'moduleA', 'moduleB', 'moduleC', '@d/moduleE', '@d/moduleF' ]
+ .value();
+ }
+
public async ensureAllDependenciesAreInstalled(projectData: IProjectData): Promise<void> {
- let installedDependencies = this.$fs.exists(this.getNodeModulesPath(projectData.projectDir)) ? this.$fs.readDirectory(this.getNodeModulesPath(projectData.projectDir)) : [];
- // Scoped dependencies are not on the root of node_modules,
- // so we have to list the contents of all directories, starting with @
- // and add them to installed dependencies, so we can apply correct comparison against package.json's dependencies.
- _(installedDependencies)
- .filter(dependencyName => _.startsWith(dependencyName, "@"))
- .each(scopedDependencyDir => {
- const contents = this.$fs.readDirectory(path.join(this.getNodeModulesPath(projectData.projectDir), scopedDependencyDir));
- installedDependencies = installedDependencies.concat(contents.map(dependencyName => `${scopedDependencyDir}/${dependencyName}`));
- });
-
+ const installedDependencies = await this.getAllInstalledDependencies(projectData);
const packageJsonContent = this.$fs.readJson(this.getPackageJsonFilePath(projectData.projectDir));
const allDependencies = _.keys(packageJsonContent.dependencies).concat(_.keys(packageJsonContent.devDependencies));
const notInstalledDependencies = _.difference(allDependencies, installedDependencies);
@@ -228,7 +243,7 @@ export class PluginsService implements IPluginsService {
}
private getPackageJsonFilePathForModule(moduleName: string, projectDir: string): string {
- return path.join(this.getNodeModulesPath(projectDir), moduleName, "package.json");
+ return require.resolve(path.join(moduleName, 'package.json'), { paths: [projectDir] });
}
private getDependencies(projectDir: string): string[] {
diff --git a/lib/tools/node-modules/node-modules-dependencies-builder.ts b/lib/tools/node-modules/node-modules-dependencies-builder.ts
index 53dd1bc..e9189ab 100644
--- a/lib/tools/node-modules/node-modules-dependencies-builder.ts
+++ b/lib/tools/node-modules/node-modules-dependencies-builder.ts
@@ -1,5 +1,5 @@
import * as path from "path";
-import { NODE_MODULES_FOLDER_NAME, PACKAGE_JSON_FILE_NAME } from "../../constants";
+import { PACKAGE_JSON_FILE_NAME } from "../../constants";
interface IDependencyDescription {
parent: IDependencyDescription;
@@ -12,7 +12,6 @@ export class NodeModulesDependenciesBuilder implements INodeModulesDependenciesB
public constructor(private $fs: IFileSystem) { }
public getProductionDependencies(projectPath: string): IDependencyData[] {
- const rootNodeModulesPath = path.join(projectPath, NODE_MODULES_FOLDER_NAME);
const projectPackageJsonPath = path.join(projectPath, PACKAGE_JSON_FILE_NAME);
const packageJsonContent = this.$fs.readJson(projectPackageJsonPath);
const dependencies = packageJsonContent && packageJsonContent.dependencies;
@@ -29,7 +28,7 @@ export class NodeModulesDependenciesBuilder implements INodeModulesDependenciesB
while (queue.length) {
const currentModule = queue.shift();
- const resolvedDependency = this.findModule(rootNodeModulesPath, currentModule, resolvedDependencies);
+ const resolvedDependency = this.findModule(currentModule);
if (resolvedDependency && !_.some(resolvedDependencies, r => r.directory === resolvedDependency.directory)) {
_.each(resolvedDependency.dependencies, d => {
@@ -53,40 +52,17 @@ export class NodeModulesDependenciesBuilder implements INodeModulesDependenciesB
return resolvedDependencies;
}
- private findModule(rootNodeModulesPath: string, depDescription: IDependencyDescription, resolvedDependencies: IDependencyData[]): IDependencyData {
- let modulePath = path.join(depDescription.parentDir, NODE_MODULES_FOLDER_NAME, depDescription.name); // node_modules/parent/node_modules/<package>
- const rootModulesPath = path.join(rootNodeModulesPath, depDescription.name);
- let depthInNodeModules = depDescription.depth;
-
- if (!this.moduleExists(modulePath)) {
-
- let moduleExists = false;
- let parent = depDescription.parent;
-
- while (parent && !moduleExists) {
- modulePath = path.join(depDescription.parent.parentDir, NODE_MODULES_FOLDER_NAME, depDescription.name);
- moduleExists = this.moduleExists(modulePath);
- if (!moduleExists) {
- parent = parent.parent;
- }
- }
-
- if (!moduleExists) {
- modulePath = rootModulesPath; // /node_modules/<package>
- if (!this.moduleExists(modulePath)) {
- return null;
- }
- }
-
- depthInNodeModules = 0;
- }
-
- if (_.some(resolvedDependencies, r => r.name === depDescription.name && r.directory === modulePath)) {
+ private findModule(depDescription: IDependencyDescription): IDependencyData {
+ try {
+ const pkgJsonName = path.join(depDescription.name, 'package.json');
+ // resolve package.json instead of module name
+ // because require.resolve('some-module') would
+ // return the file specified by "main" key of the package.json and it's module-dependent and thus impossible to exploit
+ const pkgJsonPath = require.resolve(pkgJsonName, { paths: [depDescription.parentDir] });
+ return this.getDependencyData(depDescription.name, path.dirname(pkgJsonPath), depDescription.depth);
+ } catch (e) {
return null;
-
}
-
- return this.getDependencyData(depDescription.name, modulePath, depthInNodeModules);
}
private getDependencyData(name: string, directory: string, depth: number): IDependencyData {
@@ -113,19 +89,6 @@ export class NodeModulesDependenciesBuilder implements INodeModulesDependenciesB
return null;
}
-
- private moduleExists(modulePath: string): boolean {
- try {
- let modulePathLsStat = this.$fs.getLsStats(modulePath);
- if (modulePathLsStat.isSymbolicLink()) {
- modulePathLsStat = this.$fs.getLsStats(this.$fs.realpath(modulePath));
- }
-
- return modulePathLsStat.isDirectory();
- } catch (e) {
- return false;
- }
- }
}
$injector.register("nodeModulesDependenciesBuilder", NodeModulesDependenciesBuilder);
diff --git a/package.json b/package.json
index 40b7466..4e85613 100644
--- a/package.json
+++ b/package.json
@@ -58,6 +58,7 @@
"mute-stream": "0.0.5",
"nativescript-doctor": "1.6.0",
"nativescript-preview-sdk": "0.3.0",
+ "node-walkup": "^1.1.1",
"open": "0.0.5",
"ora": "2.0.0",
"osenv": "0.1.3",
|
As a side note: I checked the source code of the |
Thank you very much for sharing the code and for your effort!! It's amazing! We really appreciate it! We're working on important updates this week and will not be able to review your code. After the release we'll consider all the possibilities and the effort we need to finish your work and make the PR. |
From @ugultopu on April 24, 2017 17:26
Is it possible to configure Nativescript CLI tool to use Yarn as the package manager?
Copied from original issue: NativeScript/NativeScript#4058
The text was updated successfully, but these errors were encountered: