Skip to content

Commit

Permalink
Further pool related fixes.
Browse files Browse the repository at this point in the history
  • Loading branch information
PaulDalek committed Jan 17, 2024
1 parent ebff476 commit c8a360a
Show file tree
Hide file tree
Showing 13 changed files with 75 additions and 50 deletions.
2 changes: 1 addition & 1 deletion .env.sample
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ EXPORT_DEFAULT_CONSTR = chart
EXPORT_DEFAULT_HEIGHT = 400
EXPORT_DEFAULT_WIDTH = 600
EXPORT_DEFAULT_SCALE = 1
EXPORT_RASTERIZATION_TIMEOUT = 1500

# Highcharts config
HIGHCHARTS_VERSION = latest
Expand Down Expand Up @@ -45,7 +46,6 @@ HIGHCHARTS_POOL_ACQUIRE_TIMEOUT = 5000
HIGHCHARTS_POOL_CREATE_TIMEOUT = 5000
HIGHCHARTS_POOL_DESTROY_TIMEOUT = 5000
HIGHCHARTS_POOL_IDLE_TIMEOUT = 30000
HIGHCHARTS_POOL_RASTERIZATION_TIMEOUT = 1500
HIGHCHARTS_POOL_CREATE_RETRY_INTERVAL = 200
HIGHCHARTS_POOL_REAPER_INTERVAL = 1000
HIGHCHARTS_POOL_BENCHMARKING = false
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ node_modules/
log/
tests/_temp
tmp/
dist/

.DS_Store
.cache
Expand Down
16 changes: 13 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,13 +262,16 @@ Loading an additional JSON configuration file can be done by using the `--loadCo
These are set as variables in your environment. They take precedence over options from the `lib/schemas/config.js` file. On Linux, use e.g. `export`.

### Export config

- `EXPORT_DEFAULT_TYPE`: The format of the file to export to. Can be jpeg, png, pdf or svg.
- `EXPORT_DEFAULT_CONSTR`: The constructor to use. Can be chart, stockChart, mapChart or ganttChart.
- `EXPORT_DEFAULT_HEIGHT`: The height of the exported chart. Overrides the option in the chart settings.
- `EXPORT_DEFAULT_WIDTH`: The width of the exported chart. Overrides the option in the chart settings.
- `EXPORT_DEFAULT_SCALE`: The scale of the exported chart. Ranges between 0.1 and 5.0.
- `EXPORT_RASTERIZATION_TIMEOUT`: The number of milliseconds to wait for rendering a webpage.

### Highcharts config

- `HIGHCHARTS_VERSION`: Highcharts version to use.
- `HIGHCHARTS_CDN`: The CDN URL of Highcharts scripts to use.
- `HIGHCHARTS_FORCE_FETCH`: Should refetch all the scripts after each server rerun.
Expand All @@ -277,21 +280,25 @@ These are set as variables in your environment. They take precedence over option
- `HIGHCHARTS_INDICATORS`: Highcharts indicators to fetch.

### Custom code config

- `HIGHCHARTS_ALLOW_CODE_EXECUTION`: If set to true, allow for the execution of arbitrary code when exporting.
- `HIGHCHARTS_ALLOW_FILE_RESOURCES`: Allow injecting resources from the filesystem. Has no effect when running as a server.

### Server config

- `HIGHCHARTS_SERVER_ENABLE`: If set to true, starts a server on 0.0.0.0.
- `HIGHCHARTS_SERVER_HOST`: The hostname of the server. Also starts a server listening on the supplied hostname.
- `HIGHCHARTS_SERVER_PORT`: The port to use for the server. Defaults to 7801.

### Server SSL config

- `HIGHCHARTS_SERVER_SSL_ENABLE`: Enables the SSL protocol.
- `HIGHCHARTS_SERVER_SSL_FORCE`: If set to true, forces the server to only serve over HTTPS.
- `HIGHCHARTS_SERVER_SSL_PORT`: The port on which to run the SSL server.
- `HIGHCHARTS_SERVER_SSL_CERT_PATH`: The path to the SSL certificate/key.

### Server rate limiting config

- `HIGHCHARTS_RATE_LIMIT_ENABLE`: Enables rate limiting.
- `HIGHCHARTS_RATE_LIMIT_MAX`: Max requests allowed in a one minute.
- `HIGHCHARTS_RATE_LIMIT_WINDOW`: The time window in minutes for rate limiting.
Expand All @@ -301,32 +308,36 @@ These are set as variables in your environment. They take precedence over option
- `HIGHCHARTS_RATE_LIMIT_SKIP_TOKEN`: Allows bypassing the rate limiter and should be provided with skipKey argument.

### Pool config

- `HIGHCHARTS_POOL_MIN_WORKERS`: The number of initial workers to spawn.
- `HIGHCHARTS_POOL_MAX_WORKERS`: The number of max workers to spawn.
- `HIGHCHARTS_POOL_WORK_LIMIT`: The pieces of work that can be performed before restarting process.
- `HIGHCHARTS_POOL_ACQUIRE_TIMEOUT`: The number of milliseconds to wait for acquiring a resource.
- `HIGHCHARTS_POOL_CREATE_TIMEOUT`: The number of milliseconds to wait for creating a resource.
- `HIGHCHARTS_POOL_DESTROY_TIMEOUT`: The number of milliseconds to wait for destroying a resource.
- `HIGHCHARTS_POOL_IDLE_TIMEOUT`: The number of milliseconds after an idle resource is destroyed.
- `HIGHCHARTS_POOL_RASTERIZATION_TIMEOUT`: The number of milliseconds to wait for rendering a webpage.
- `HIGHCHARTS_POOL_CREATE_RETRY_INTERVAL`: The number of milliseconds after the create process is retried in case of fail.
- `HIGHCHARTS_POOL_REAPER_INTERVAL`: The number of milliseconds after the check for idle resources to destroy is triggered.
- `HIGHCHARTS_POOL_BENCHMARKING`: Enable benchmarking.
- `HIGHCHARTS_POOL_LISTEN_TO_PROCESS_EXITS`: Set to false in order to skip attaching process.exit handlers.

### Logging config

- `HIGHCHARTS_LOG_LEVEL`: The log level (0: silent, 1: error, 2: warning, 3: notice, 4: verbose).
- `HIGHCHARTS_LOG_FILE`: A name of a log file. The --logDest also needs to be set to enable file logging.
- `HIGHCHARTS_LOG_DEST`: The path to store log files. Also enables file logging.

### UI config

- `HIGHCHARTS_UI_ENABLE`: Enables the UI for the export server.
- `HIGHCHARTS_UI_ROUTE`: The route to attach the UI to.

### Other config

- `HIGHCHARTS_NO_LOGO`: Skip printing the logo on a startup. Will be replaced by a simple text.

### Proxy config

- `PROXY_SERVER_HOST`: The host of the proxy server to use if exists.
- `PROXY_SERVER_PORT`: The port of the proxy server to use if exists.
- `PROXY_SERVER_TIMEOUT`: The timeout for the proxy server to use if exists.
Expand All @@ -350,6 +361,7 @@ _Available options:_
- `--globalOptions`: A stringified JSON or a filename with options to be passed into the Highcharts.setOptions (defaults to `false`).
- `--themeOptions`: A stringified JSON or a filename with theme options to be passed into the Highcharts.setOptions (defaults to `false`).
- `--batch`: Starts a batch job. A string that contains input/output pairs: "in=out;in=out;.." (defaults to `false`).
- `--rasterizationTimeout`: The number of milliseconds to wait for rendering a webpage (defaults to `1500`).
- `--allowCodeExecution`: If set to true, allow for the execution of arbitrary code when exporting (defaults to `false`).
- `--allowFileResources`: Allow injecting resources from the filesystem. Has no effect when running as a server (defaults to `true`).
- `--customCode`: Custom code to be called before chart initialization. Can be a function, a code that will be wrapped within a function or a filename with the js extension (defaults to `false`).
Expand All @@ -375,7 +387,6 @@ _Available options:_
- `--createTimeout`: The number of milliseconds to wait for creating a resource (defaults to `5000`).
- `--destroyTimeout`: The number of milliseconds to wait for destroying a resource (defaults to `5000`).
- `--idleTimeout`: The number of milliseconds after an idle resource is destroyed (defaults to `30000`).
- `--rasterizationTimeout`: The number of milliseconds to wait for rendering a webpage (defaults to `1500`).
- `--createRetryInterval`: The number of milliseconds after the create process is retried in case of fail (defaults to `200`).
- `--reaperInterval`: The number of milliseconds after the check for idle resources to destroy is triggered (defaults to `1000`).
- `--benchmarking`: Enable benchmarking (defaults to `true`).
Expand Down Expand Up @@ -636,7 +647,6 @@ This package supports both CommonJS and ES modules.
- `createTimeout` (default 5000) - The maximum allowed time for each resource create, in milliseconds.
- `destroyTimeout` (default 5000) - The maximum allowed time for each resource destroy, in milliseconds.
- `idleTimeout` (default 30000) - The maximum allowed time after an idle resource is destroyed, in milliseconds.
- `rasterizationTimeout` (default 1500) - The number of milliseconds to wait for rendering a webpage.
- `createRetryInterval` (default 200) - The number of milliseconds after the create process is retried in case of fail.
- `reaperInterval` (default 1000) - The number of milliseconds after the check for idle resources to destroy is triggered.
- `benchmarking` (default false) - Enable benchmarking.
Expand Down
2 changes: 0 additions & 2 deletions dist/index.cjs

This file was deleted.

2 changes: 0 additions & 2 deletions dist/index.esm.js

This file was deleted.

1 change: 0 additions & 1 deletion dist/index.esm.js.map

This file was deleted.

25 changes: 19 additions & 6 deletions lib/browser.js
Original file line number Diff line number Diff line change
Expand Up @@ -104,18 +104,31 @@ export const newPage = async () => {

const page = await browser.newPage();

// Disable cache
await page.setCacheEnabled(false);

// Set the content
await setPageContent(page);
return page;
};

export const clearPage = async (page) => {
export const clearPage = async (page, hardReset = false) => {
try {
// Navigate to about:blank
await page.goto('about:blank');

// Set the content and and scripts again
await setPageContent(page);
if (hardReset) {
// Navigate to about:blank
await page.goto('about:blank');

// Set the content and and scripts again
await setPageContent(page);
} else {
// Clear body content
await page.$eval(
'body',
(body) =>
(body.innerHTML =
'<div id="chart-container"><div id="container"></div></div>')
);
}
} catch (error) {
log(3, '[browser] Could not clear page');
}
Expand Down
5 changes: 3 additions & 2 deletions lib/export.js
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,8 @@ const getClipRegion = (page) =>
* @param {string} type - The type of a result image.
* @param {string} encoding - The type of encoding used.
* @param {string} clip - The clip region.
* @param {number} rasterizationTimeout - The rasterization timeout in milliseconds.
* @param {number} rasterizationTimeout - The rasterization timeout
* in milliseconds.
* @returns {string} - A string representation of a screenshot.
*/
const createImage = async (page, type, encoding, clip, rasterizationTimeout) =>
Expand Down Expand Up @@ -413,7 +414,7 @@ export default async (page, chart, options) => {
x,
y
},
options.pool.rasterizationTimeout
exportOptions.rasterizationTimeout
);
} else if (exportOptions.type === 'pdf') {
// PDF
Expand Down
6 changes: 4 additions & 2 deletions lib/pool.js
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ const factory = {
}

// Clear page
await clearPage(workerHandle.page);
await clearPage(workerHandle.page, false);
return true;
},

Expand Down Expand Up @@ -202,7 +202,9 @@ export const init = async (config) => {
);
});

pool.on('release', (resource) => {
pool.on('release', async (resource) => {
// Clear page
await clearPage(resource.page, false);
log(4, `[pool] Releasing a worker of an id ${resource.id}`);
});

Expand Down
24 changes: 12 additions & 12 deletions lib/schemas/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,12 @@ export const defaultConfig = {
type: 'string',
description:
'Starts a batch job. A string that contains input/output pairs: "in=out;in=out;..".'
},
rasterizationTimeout: {
envLink: 'EXPORT_RASTERIZATION_TIMEOUT',
value: 1500,
type: 'number',
description: 'The number of milliseconds to wait for rendering a webpage.'
}
},
customCode: {
Expand Down Expand Up @@ -420,12 +426,6 @@ export const defaultConfig = {
description:
'The number of milliseconds after an idle resource is destroyed.'
},
rasterizationTimeout: {
envLink: 'HIGHCHARTS_POOL_RASTERIZATION_TIMEOUT',
value: 1500,
type: 'number',
description: 'The number of milliseconds to wait for rendering a webpage.'
},
createRetryInterval: {
envLink: 'HIGHCHARTS_POOL_CREATE_RETRY_INTERVAL',
value: 200,
Expand Down Expand Up @@ -593,6 +593,12 @@ export const promptsConfig = {
initial: defaultConfig.export.defaultScale.value,
min: 0.1,
max: 5
},
{
type: 'number',
name: 'rasterizationTimeout',
message: 'The number of milliseconds to wait for rendering a webpage',
initial: defaultConfig.export.rasterizationTimeout.value
}
],
customCode: [
Expand Down Expand Up @@ -742,12 +748,6 @@ export const promptsConfig = {
message: 'The number of milliseconds after an idle resource is destroyed',
initial: defaultConfig.pool.idleTimeout.value
},
{
type: 'number',
name: 'rasterizationTimeout',
message: 'The number of milliseconds to wait for rendering a webpage',
initial: defaultConfig.pool.rasterizationTimeout.value
},
{
type: 'number',
name: 'createRetryInterval',
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"author": "Highsoft AS <support@highcharts.com> (http://www.highcharts.com/about)",
"license": "MIT",
"type": "module",
"version": "3.0.5",
"version": "3.1.0",
"main": "dist/index.esm.js",
"exports": {
".": {
Expand Down
35 changes: 19 additions & 16 deletions templates/template.html
Original file line number Diff line number Diff line change
@@ -1,16 +1,21 @@
<!DOCTYPE html>
<html lang='en-US'>
<!doctype html>
<html lang="en-US">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Highcharts Export</title>
<style>
html, body {
html,
body {
margin: 0;
padding: 0;
box-sizing: border-box;
}

#table-div, #sliders, #datatable, #controls, .ld-row {
#table-div,
#sliders,
#datatable,
#controls,
.ld-row {
display: none;
height: 0;
}
Expand All @@ -21,7 +26,8 @@
overflow: auto;
}

#chart-container > figure, div {
#chart-container > figure,
div {
margin-top: 0 !important;
margin-bottom: 0 !important;
}
Expand Down Expand Up @@ -89,9 +95,13 @@

// Add flag to know if chart render has been called.
if (!window.onHighchartsRender) {
window.onHighchartsRender = Highcharts.addEvent(this, 'render', () => {
window.isRenderComplete = true;
});
window.onHighchartsRender = Highcharts.addEvent(
this,
'render',
() => {
window.isRenderComplete = true;
}
);
}

proceed.apply(this, [userOptions, cb]);
Expand Down Expand Up @@ -155,13 +165,6 @@
// Get the chart section of options
const { chart } = chartOptions;

// Clear previous state
document.body.innerHTML = `
<div id="chart-container">
<div id="container"></div>
</div>
`;

// Create a separate object for a potential setOptions usages in order to
// prevent from polluting other exports that can happen on the same page
Highcharts.setOptionsObj = Highcharts.merge(
Expand Down

0 comments on commit c8a360a

Please sign in to comment.