tns run android fails with private npm registry

[facetious]

Environment
Provide version numbers for the following components (information can be retrieved by running tns info in your project folder or by inspecting the package.json of the project):
tns --version: 5.1.0
tns info:
✔ Component nativescript has 5.1.0 version and is up to date.
✔ Component tns-core-modules has 5.1.1 version and is up to date.
✔ Component tns-android has 5.1.0 version and is up to date.

Describe the bug
The first step in using a private npm mirror was completed here: #3866
The next step (properly authenticating) was not completed. As a result, our private mirror (which requires authentication that is specified in the .npmrc file) is not successfully reachable.

To Reproduce
Set up a private npm server that requires authentication. Place your authentication credentials in .npmrc (auth-always=true, email= ..., _auth= ...).
tns build android --bundle --env.aot --env.uglify --log trace
Observe:

httpRequest: Sending:
[no content]
httpRequest: Done. code = 401
Request was unsuccessful. Server returned:  
Failed to get error from http request:  SyntaxError: Unexpected end of JSON input

Expected behavior
It builds.

Sample project
Any project with 5.0 or 5.1 will do.

[facetious]

The previous versions of NativeScript were happy to use npm properly. Why is 5.0+ reimplementing it?

[Fatme]

@facetious,

Can you send us the full log from tns build android --bundle --env.aot --env.uglify --log trace command?

[mikaelkalt]

Seems like we're facing the same issue - I cannot build anymore the android app after upgrading to 5.0+!

When I run tns run android --log trace with or without --bundle I get the following error:

Exec npm config get registry 
 stdout: https://nexus.company.com/repository/npm-group/
 
 stderr: 
Trying to get data from npm registry for package tns-android, url is: https://nexus.company.com/repository/npm-group/tns-android
User-Agent: tnsCLI/5.1.0 (Node.js 8.9.4; darwin; x64)
httpRequest: { url: 'https://nexus.company.com/repository/npm-group/tns-android',
  method: 'GET',
  proto: 'https',
  host: 'nexus.company.com',
  port: null,
  path: '/repository/npm-group/tns-android',
  headers: 
   { Accept: 'application/json; charset=UTF-8, */*;q=0.8',
     'User-Agent': 'tnsCLI/5.1.0 (Node.js 8.9.4; darwin; x64)',
     'Accept-Encoding': 'gzip,deflate' },
  encoding: null,
  followAllRedirects: true }
httpRequest: Sending:
[no content]
httpRequest: Done. code = 401
Request was unsuccessful. Server returned:  
Failed to get error from http request:  SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at HttpClient.getErrorMessage (/Users/user/.nvm/versions/node/v8.9.4/lib/node_modules/nativescript/lib/common/http-client.js:235:34)
    at IncomingMessage.responseStream.on (/Users/user/.nvm/versions/node/v8.9.4/lib/node_modules/nativescript/lib/common/http-client.js:180:59)
    at emitNone (events.js:111:20)
    at IncomingMessage.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1055:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
Unable to apply changes on device: emulator-5554. Error is: The server returned unexpected response: .
Will emit event liveSyncError with data { error: 
   { Error: The server returned unexpected response: 
    at IncomingMessage.responseStream.on (/Users/user/.nvm/versions/node/v8.9.4/lib/node_modules/nativescript/lib/common/http-client.js:181:45)
    at emitNone (events.js:111:20)
    at IncomingMessage.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1055:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
     response: 
      IncomingMessage {
        _readableState: [Object],
        readable: false,
        domain: null,
        _events: [Object],
        _eventsCount: 3,
        _maxListeners: undefined,
        socket: [Object],
        connection: [Object],
        httpVersionMajor: 1,
        httpVersionMinor: 1,
        httpVersion: '1.1',
        complete: true,
        headers: [Object],
        rawHeaders: [Array],
        trailers: {},
        rawTrailers: [],
        upgrade: false,
        url: '',
        method: null,
        statusCode: 401,
        statusMessage: 'Unauthorized',
        client: [Object],
        _consuming: true,
        _dumped: true,
        req: [Object],
        request: [Object],
        toJSON: [Function: responseToJSON],
        caseless: [Object],
        read: [Function] },
     body: '' },
  deviceIdentifier: 'emulator-5554',
  projectDir: '/Users/user/Documents/Projects/customer/app',
  applicationIdentifier: 'de.customer.fotoapplication' }

npm install works fine..

[madmas]

I can confirm this issue, also validated with Node 10.15.0 and NPM 6.5.0 .
"tns doctor" runs and has nothing to complain.

@Fatme the output of that command is pretty long ad contains too many details to anonymize, can I supply it on another channel?

It appears that indeed nativescript/lib/common/http-client.jsis not using the credential to access the npm registry. Happy for any hint how I can help to solve this issue.

[rosen-vladimirov]

Hey @madmas , you can upload your files somewhere and share them with me in the NativeScript Community Slack - my username is vladimirov

[madmas]

Done ;-)

[rosen-vladimirov]

Hey @madmas, thanks for the provided information.

It looks like some of your dependencies are not built for Android and during build operation CLI tries to build .aar file for each of them.

During the process of building the .aar file for a plugin, CLI tries to get information which gradle versions to use, based on your version of tns-android in your package.json. Once CLI founds your tns-android version (for example 5.1.0), CLI tries to get the information for this package from npm registry. In the package.json (that we access from the npm registry), CLI searches for specific information that shows which gradle version to use for building the plugin

One way to workaround this is to prebuilt the .aar on your own in node_modules.

Another way is to change this line in your local CLI installation:

nativescript-cli/lib/node-package-manager.ts

Line 111 in f83c95e

const registry = await this.$childProcess.exec(`npm config get registry`);

From:
const registry = yield this.$childProcess.exec(`npm config get registry`);

To:
const registry = "http://registry.npmjs.org";

In case you are able to access the official registry, CLI will use it to get the data for tns-android package. For all other operations, CLI will spawn npm, which will use your npmrc file.

We'll investigate this issue further and update this issue once we have more information.

[facetious]

@Fatme I think you will find that the more appropriate way to do this is to leverage the existing npm tool.

For example, if you call npm view tns-android --json, you will receive a json blob back (from the appropriate server using the .npmrc correctly) that contains the chunk your CLI is after:

"gradle": { "version": "4.10.2", "android": "3.2.1" }

I insist that you remove all manual circumventing of npm. It will make your future maintenance significantly easier.

[Fatme]

@facetious,

I totally agree we shouldn't use any manual circumventing of npm.

Actually our first implementation was with npm view. Unfortunately, there were too much problems caused by npm view. Some versions of npm have bugs and the output from npm view --json isn't in the expected format. It was too hard to handle all the situations and decided to make manually the http request. Obviously, there are another problems caused by this approach.

We're working on investigating the issue and searching for a solution. We''ll update you when we have more information.

[facetious]

@Fatme I would be curious to know which versions of npm you are having difficulty with. This one dates back to 2015-10-29 and seems to still provide the functionality you desire.

 :~/npmtest$ node_modules/.bin/npm --version
3.3.6
 :~/npmtest$ node_modules/.bin/npm view tns-android@latest gradle
{ version: '4.10.2', android: '3.2.1' }
 :~/npmtest$ node_modules/.bin/npm view tns-android@4.2 gradle
{ version: '4.4', android: '3.1.3' }

[Fatme]

Hey @facetious, @madmas,

We discussed the problem and based on your input we decided to use npm view command. We'll fallback to the current code in case when npm view returns data in unexpected format.
The fix is already merged in master branch so you can test it on your side using npm i -g nativescript@next.
Let us know if it works for you.

[madmas]

@Fatme thanks a lot for the fix, it works like a charm for us! looking forward to the 5.2 release :)