Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: reuse local du.zip #56

Merged
merged 18 commits into from
Aug 10, 2023
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
.idea/
node_modules/
bin/
.nyc_output/
17 changes: 13 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ Node CLI or module to calculate folder size.

It uses:

- [Sysinternals DU](https://docs.microsoft.com/en-us/sysinternals/downloads/du) on Windows, automatically downloaded at installation time because the license does not allow redistribution. See below about specifying the download location.
- [Sysinternals DU](https://docs.microsoft.com/en-us/sysinternals/downloads/du) on Windows, automatically downloaded at
installation time because the license does not allow redistribution. See below about specifying the download location.
- native `du` on other platforms

## Installation
Expand Down Expand Up @@ -57,9 +58,17 @@ fast-folder-size .

By default the Sysinternals DU.zip is downloaded from https://download.sysinternals.com/files/DU.zip.

If you need to change this, e.g. to download from an internal package repository,
set the **FAST_FOLDER_SIZE_DU_ZIP_LOCATION** environment variable. For example:
If you need to change this, e.g. to download from an internal package repository
or re-use an existing du.zip, you can set the **FAST_FOLDER_SIZE_DU_ZIP_LOCATION** environment variable.

For example:

```shell
export FAST_FOLDER_SIZE_DU_ZIP_LOCATION="https://your.internal.repository/DU.zip"
```

or

```shell
export FAST_FOLDER_SIZE_DU_ZIP_LOCATION=https://your.internal.repository/DU.zip
export FAST_FOLDER_SIZE_DU_ZIP_LOCATION="D://download/du.zip"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why double slash after the drive name? I assume we're targeting Windows in this case, so shouldn't the path separator be backslash instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's personal habit. I tested the C://**/du.zip and C:\**\du.zip, they both pass the test.

Should we add another example like:

or

export FAST_FOLDER_SIZE_DU_ZIP_LOCATION="D:\download\du.zip"

or just replace "D://download/du.zip" with "D:\download\du.zip" ?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's just make sure this works on the windows terminal and not an emulator

```
101 changes: 69 additions & 32 deletions get-sysinternals-du.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,45 +5,82 @@ const path = require('path')
const decompress = require('decompress')
const { HttpsProxyAgent } = require('https-proxy-agent')

// Only run for Windows
if (process.platform !== 'win32') {
process.exit(0)
}
exports.workspace = __dirname
DevDengChao marked this conversation as resolved.
Show resolved Hide resolved

// check if du is already installed
if (
fs.existsSync(
path.join(__dirname, 'bin', `du${process.arch === 'x64' ? '64' : ''}.exe`)
)
) {
process.exit(0)
exports.onDuZipDownloaded = function (tempFilePath) {
decompress(tempFilePath, path.join(exports.workspace, 'bin'))
}

const duZipLocation =
process.env.FAST_FOLDER_SIZE_DU_ZIP_LOCATION ||
'https://download.sysinternals.com/files/DU.zip'
exports.downloadDuZip = function (mirror) {
const duZipLocation =
mirror || 'https://download.sysinternals.com/files/DU.zip'

// checks for proxy variables in user environment
const proxyAddress =
process.env.HTTPS_PROXY ||
process.env.https_proxy ||
process.env.HTTP_PROXY ||
process.env.http_proxy
// checks for proxy variables in user environment
const proxyAddress =
process.env.HTTPS_PROXY ||
process.env.https_proxy ||
process.env.HTTP_PROXY ||
process.env.http_proxy

if (proxyAddress) {
const agent = new HttpsProxyAgent(proxyAddress)
if (proxyAddress) {
https.globalAgent = new HttpsProxyAgent(proxyAddress)
}

https.globalAgent = agent
}
console.log(`downloading du.zip from ${duZipLocation}`)
if (!mirror) {
console.log(
`if you have trouble while downloading, try set process.env.FAST_FOLDER_SIZE_DU_ZIP_LOCATION to a proper mirror or local file path`
)
}

https.get(duZipLocation, function (res) {
const tempFilePath = path.join(os.tmpdir(), 'du.zip')
https.get(duZipLocation, function (res) {
const tempFilePath = path.join(os.tmpdir(), 'du.zip')

const fileStream = fs.createWriteStream(tempFilePath)
res.pipe(fileStream)
const fileStream = fs.createWriteStream(tempFilePath)
res.pipe(fileStream)

fileStream.on('finish', function () {
fileStream.close()
decompress(tempFilePath, path.join(__dirname, 'bin'))
fileStream.on('finish', function () {
fileStream.close()
exports.onDuZipDownloaded(tempFilePath)
})
})
})
}

exports.default = function () {
// Only run for Windows
if (process.platform !== 'win32') {
return
}

// check if du is already installed
let duBinFilename = `du${process.arch === 'x64' ? '64' : ''}.exe`
let defaultDuBinPath = path.join(exports.workspace, 'bin', duBinFilename)
DevDengChao marked this conversation as resolved.
Show resolved Hide resolved

if (fs.existsSync(defaultDuBinPath)) {
console.log(`${duBinFilename} found at ${defaultDuBinPath}`)
return
} else {
DevDengChao marked this conversation as resolved.
Show resolved Hide resolved
console.log(`${duBinFilename} not found at ${defaultDuBinPath}`)
}

let mirrorOrCache = process.env.FAST_FOLDER_SIZE_DU_ZIP_LOCATION

if (
!mirrorOrCache ||
mirrorOrCache.startsWith('http://') ||
mirrorOrCache.startsWith('https://')
) {
exports.downloadDuZip(mirrorOrCache)
} else if (fs.existsSync(mirrorOrCache)) {
exports.onDuZipDownloaded(mirrorOrCache)
} else {
let message = `du.zip not found at ${mirrorOrCache}`
// this will result the process to exit with code 1
throw Error(message)
}
DevDengChao marked this conversation as resolved.
Show resolved Hide resolved
}

// only auto execute default() function when its invoked directly
if (require.main === module) {
exports.default()
}
64 changes: 64 additions & 0 deletions get-sysinternals-du.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
if (process.platform !== 'win32') {
// these test cases are only for win32
return
}

const { test, beforeEach } = require('tap')
const path = require('path')
const os = require('os')
const fs = require('fs')
const subject = require('./get-sysinternals-du.js')

let workspace = path.join(os.tmpdir(), 'fast-folder-size-playground')
beforeEach(() => {
if (fs.existsSync(workspace)) fs.rmSync(workspace, { recursive: true })
fs.mkdirSync(workspace)
subject.workspace = workspace
})

test('it can use local file path as process.env.FAST_FOLDER_SIZE_DU_ZIP_LOCATION', t => {
let dummyDuZipPath = path.join(workspace, 'dummy-du.zip')
fs.writeFileSync(dummyDuZipPath, '')
process.env.FAST_FOLDER_SIZE_DU_ZIP_LOCATION = dummyDuZipPath

subject.onDuZipDownloaded = function (tempFilePath) {
t.equal(dummyDuZipPath, tempFilePath)
t.end()
}

subject.default()
})
test('it cannot use non-exists local file path as process.env.FAST_FOLDER_SIZE_DU_ZIP_LOCATION', t => {
process.env.FAST_FOLDER_SIZE_DU_ZIP_LOCATION = path.join(
workspace,
'non-exists-dummy-du.zip'
)

t.throws(subject.default, error => {
return error.message.startsWith('du.zip not found at')
})
t.end()
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when you don't need to assert asynchronously in callbacks, please write the test function as an async function so you don't have to remember calling t.end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Once I added the async keyword, eslint shows an error on the parameter t: ESLint: Parsing error: Unexpected token t.
Everything looks fine to me as I'm following the doc https://node-tap.org/docs/api/promises/ , do you know how to fix this ?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably just an eslint configuration to support esm, should be easy to address

})

test('it can use http(s) url as process.env.FAST_FOLDER_SIZE_DU_ZIP_LOCATION', t => {
let dummyUrl = 'https://non-exists.localhost/du.zip'
process.env.FAST_FOLDER_SIZE_DU_ZIP_LOCATION = dummyUrl

subject.downloadDuZip = function (mirror) {
t.equal(dummyUrl, mirror)
DevDengChao marked this conversation as resolved.
Show resolved Hide resolved
t.end()
}

subject.default()
})

test('when process.env.FAST_FOLDER_SIZE_DU_ZIP_LOCATION not found, then download it directly', t => {
delete process.env.FAST_FOLDER_SIZE_DU_ZIP_LOCATION

subject.downloadDuZip = function (mirror) {
t.equal(undefined, mirror)
t.end()
}

subject.default()
})