Skip to content

Commit

Permalink
feat: implement the directory option for the cacheTransform option
Browse files Browse the repository at this point in the history
BREAKING CHANGE: the `cacheTransform` option should have only `directory` and `keys` properties when it is an object
  • Loading branch information
evilebottnawi authored May 15, 2020
1 parent bebafcf commit 29254e3
Show file tree
Hide file tree
Showing 8 changed files with 559 additions and 140 deletions.
160 changes: 145 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,19 +74,19 @@ module.exports = {

### Patterns

| Name | Type | Default | Description |
| :-------------------------------------: | :-----------------: | :---------------------------------------------: | :---------------------------------------------------------------------------------------------------- |
| [`from`](#from) | `{String}` | `undefined` | Glob or path from where we сopy files. |
| [`to`](#to) | `{String}` | `compiler.options.output` | Output path. |
| [`context`](#context) | `{String}` | `options.context \|\| compiler.options.context` | A path that determines how to interpret the `from` path. |
| [`globOptions`](#globoptions) | `{Object}` | `undefined` | [Options][glob-options] passed to the glob pattern matching library including `ignore` option. |
| [`toType`](#totype) | `{String}` | `undefined` | Determinate what is `to` option - directory, file or template. |
| [`force`](#force) | `{Boolean}` | `false` | Overwrites files already in `compilation.assets` (usually added by other plugins/loaders). |
| [`flatten`](#flatten) | `{Boolean}` | `false` | Removes all directory references and only copies file names. |
| [`transform`](#transform) | `{Function}` | `undefined` | Allows to modify the file contents. |
| [`cacheTransform`](#cacheTransform) | `{Boolean\|Object}` | `false` | Enable `transform` caching. You can use `{ cache: { key: 'my-cache-key' } }` to invalidate the cache. |
| [`transformPath`](#transformpath) | `{Function}` | `undefined` | Allows to modify the writing path. |
| [`noErrorOnMissing`](#noerroronmissing) | `{Boolean}` | `false` | Doesn't generate an error on missing file(s). |
| Name | Type | Default | Description |
| :-------------------------------------: | :-------------------------: | :---------------------------------------------: | :---------------------------------------------------------------------------------------------------- |
| [`from`](#from) | `{String}` | `undefined` | Glob or path from where we сopy files. |
| [`to`](#to) | `{String}` | `compiler.options.output` | Output path. |
| [`context`](#context) | `{String}` | `options.context \|\| compiler.options.context` | A path that determines how to interpret the `from` path. |
| [`globOptions`](#globoptions) | `{Object}` | `undefined` | [Options][glob-options] passed to the glob pattern matching library including `ignore` option. |
| [`toType`](#totype) | `{String}` | `undefined` | Determinate what is `to` option - directory, file or template. |
| [`force`](#force) | `{Boolean}` | `false` | Overwrites files already in `compilation.assets` (usually added by other plugins/loaders). |
| [`flatten`](#flatten) | `{Boolean}` | `false` | Removes all directory references and only copies file names. |
| [`transform`](#transform) | `{Function}` | `undefined` | Allows to modify the file contents. |
| [`cacheTransform`](#cacheTransform) | `{Boolean\|String\|Object}` | `false` | Enable `transform` caching. You can use `{ cache: { key: 'my-cache-key' } }` to invalidate the cache. |
| [`transformPath`](#transformpath) | `{Function}` | `undefined` | Allows to modify the writing path. |
| [`noErrorOnMissing`](#noerroronmissing) | `{Boolean}` | `false` | Doesn't generate an error on missing file(s). |

#### `from`

Expand Down Expand Up @@ -456,12 +456,16 @@ module.exports = {

#### `cacheTransform`

Type: `Boolean|Object`
Type: `Boolean|String|Object`
Default: `false`

Enable/disable `transform` caching. You can use `{ cacheTransform: { key: 'my-cache-key' } }` to invalidate the cache.
Enable/disable and configure caching.
Default path to cache directory: `node_modules/.cache/copy-webpack-plugin`.

##### `Boolean`

Enables/Disable `transform` caching.

**webpack.config.js**

```js
Expand All @@ -483,6 +487,132 @@ module.exports = {
};
```

##### `String`

Enables `transform` caching and setup cache directory.

**webpack.config.js**

```js
module.exports = {
plugins: [
new CopyPlugin({
patterns: [
{
from: 'src/*.png',
to: 'dest/',
transform(content, path) {
return optimize(content);
},
// Should be absolute
cacheTransform: path.resolve(__dirname, 'cache-directory'),
},
],
}),
],
};
```

##### `Object`

Enables `transform` caching and setup cache directory and invalidation keys.

**webpack.config.js**

```js
module.exports = {
plugins: [
new CopyPlugin({
patterns: [
{
from: 'src/*.png',
to: 'dest/',
transform(content, path) {
return optimize(content);
},
cacheTransform: {
directory: path.resolve(__dirname, 'cache-directory'),
keys: {
// May be useful for invalidating cache based on external values
// For example, you can invalid cache based on `process.version` - { node: process.version }
key: 'value',
},
},
},
],
}),
],
};
```

You can setup invalidation keys using a function.

Simple function:

**webpack.config.js**

```js
module.exports = {
plugins: [
new CopyPlugin({
patterns: [
{
from: 'src/*.png',
to: 'dest/',
transform(content, path) {
return optimize(content);
},
cacheTransform: {
directory: path.resolve(__dirname, 'cache-directory'),
keys: (defaultCacheKeys, absoluteFrom) => {
const keys = getCustomCacheInvalidationKeysSync();

return {
...defaultCacheKeys,
keys,
};
},
},
},
],
}),
],
};
```

Async function:

**webpack.config.js**

```js
module.exports = {
plugins: [
new CopyPlugin({
patterns: [
{
from: 'src/*.png',
to: 'dest/',
transform(content, path) {
return optimize(content);
},
cacheTransform: {
directory: path.resolve(__dirname, 'cache-directory'),
keys: async (defaultCacheKeys, absoluteFrom) => {
const keys = await getCustomCacheInvalidationKeysAsync();

return {
...defaultCacheKeys,
keys,
};
},
},
},
],
}),
],
};
```

#### `transformPath`

Type: `Function`
Expand Down
23 changes: 22 additions & 1 deletion src/options.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,28 @@
"type": "boolean"
},
{
"type": "object"
"type": "string"
},
{
"type": "object",
"additionalProperties": false,
"properties": {
"directory": {
"type": "string",
"absolutePath": true
},
"keys": {
"anyOf": [
{
"type": "object",
"additionalProperties": true
},
{
"instanceof": "Function"
}
]
}
}
}
]
},
Expand Down
42 changes: 27 additions & 15 deletions src/postProcessPattern.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import normalizePath from 'normalize-path';

import { RawSource } from 'webpack-sources';

import { name, version } from '../package.json';
import { version } from '../package.json';

import { stat, readFile } from './utils/promisify';

Expand Down Expand Up @@ -63,22 +63,33 @@ export default async function postProcessPattern(globalRef, pattern, file) {
logger.log(`transforming content for '${file.absoluteFrom}'`);

if (pattern.cacheTransform) {
if (!globalRef.cacheDir) {
globalRef.cacheDir =
findCacheDir({ name: 'copy-webpack-plugin' }) || os.tmpdir();
const cacheDirectory = pattern.cacheTransform.directory
? pattern.cacheTransform.directory
: typeof pattern.cacheTransform === 'string'
? pattern.cacheTransform
: findCacheDir({ name: 'copy-webpack-plugin' }) || os.tmpdir();
let defaultCacheKeys = {
version,
transform: pattern.transform,
contentHash: crypto.createHash('md4').update(content).digest('hex'),
};

if (typeof pattern.cacheTransform.keys === 'function') {
defaultCacheKeys = await pattern.cacheTransform.keys(
defaultCacheKeys,
file.absoluteFrom
);
} else {
defaultCacheKeys = {
...defaultCacheKeys,
...pattern.cacheTransform.keys,
};
}

const cacheKey = pattern.cacheTransform.key
? pattern.cacheTransform.key
: serialize({
name,
version,
pattern,
hash: crypto.createHash('md4').update(content).digest('hex'),
});
const cacheKeys = serialize(defaultCacheKeys);

try {
const result = await cacache.get(globalRef.cacheDir, cacheKey);
const result = await cacache.get(cacheDirectory, cacheKeys);

logger.debug(
`getting cached transformation for '${file.absoluteFrom}'`
Expand All @@ -91,7 +102,7 @@ export default async function postProcessPattern(globalRef, pattern, file) {
logger.debug(`caching transformation for '${file.absoluteFrom}'`);

content = await cacache
.put(globalRef.cacheDir, cacheKey, content)
.put(cacheDirectory, cacheKeys, content)
.then(() => content);
}
} else {
Expand Down Expand Up @@ -138,14 +149,15 @@ export default async function postProcessPattern(globalRef, pattern, file) {
const source = new RawSource(content);

// For old version webpack 4
/* istanbul ignore if */
if (typeof compilation.emitAsset !== 'function') {
compilation.assets[targetPath] = source;

return;
}

if (compilation.getAsset(targetPath)) {
if (file.force) {
if (pattern.force) {
logger.log(
`force updating '${file.webpackTo}' to compilation assets from '${file.absoluteFrom}'`
);
Expand Down
5 changes: 1 addition & 4 deletions src/processPattern.js
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,7 @@ export default async function processPattern(globalRef, pattern) {
}

return paths.map((from) => {
const file = {
force: pattern.force,
absoluteFrom: path.resolve(pattern.context, from),
};
const file = { absoluteFrom: path.resolve(pattern.context, from) };

file.relativeFrom = path.relative(pattern.context, file.absoluteFrom);

Expand Down
8 changes: 4 additions & 4 deletions src/utils/createPatternGlob.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,13 @@ function createPatternGlob(pattern, globalRef) {
const { logger, compilation } = globalRef;

// eslint-disable-next-line no-param-reassign
pattern.globOptions = Object.assign(
{
pattern.globOptions = {
...{
cwd: pattern.context,
followSymbolicLinks: true,
},
pattern.globOptions || {}
);
...(pattern.globOptions || {}),
};

switch (pattern.fromType) {
case 'dir':
Expand Down
11 changes: 9 additions & 2 deletions test/__snapshots__/validate-options.test.js.snap
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ exports[`validate options should throw an error on the "patterns" option with "[
- options.patterns[0].from should be an non-empty string."
`;
exports[`validate options should throw an error on the "patterns" option with "[{"from":"test.txt","to":"dir","context":"context","cacheTransform":{"foo":"bar"}}]" value 1`] = `
"Invalid options object. Copy Plugin has been initialized using an options object that does not match the API schema.
- options.patterns[0].cacheTransform has an unknown property 'foo'. These properties are valid:
object { directory?, keys? }"
`;
exports[`validate options should throw an error on the "patterns" option with "[{"from":"test.txt","to":"dir","context":"context","flatten":"true"}]" value 1`] = `
"Invalid options object. Copy Plugin has been initialized using an options object that does not match the API schema.
- options.patterns[0].flatten should be a boolean."
Expand Down Expand Up @@ -69,11 +75,12 @@ exports[`validate options should throw an error on the "patterns" option with "[
non-empty string | object { from, to?, context?, globOptions?, toType?, force?, flatten?, transform?, cacheTransform?, transformPath?, noErrorOnMissing? }
Details:
* options.patterns[0].cacheTransform should be one of these:
boolean | object { }
boolean | string | object { directory?, keys? }
Details:
* options.patterns[0].cacheTransform should be a boolean.
* options.patterns[0].cacheTransform should be a string.
* options.patterns[0].cacheTransform should be an object:
object { }"
object { directory?, keys? }"
`;
exports[`validate options should throw an error on the "patterns" option with "[{"from":"test.txt","to":"dir","context":true}]" value 1`] = `
Expand Down
Loading

0 comments on commit 29254e3

Please sign in to comment.