Implement better terminal output mode

README.md

@@ -1,74 +1,159 @@
-# Benchpress 
+# Benchpress
 
-Benchmarking TS library with V8 warmup and self-denoising for the most accurate results. 
+A modular benchmarking library with V8 warmup and cpu/ram denoising for the most accurate and consistent results.
 
 ## Features
 
-- Waits until V8 optimizations kick in
-- Gets rid of noise caused by the library itself 
-- Uses high resolution time in nanoseconds for more accurate op/s
-- Reuses `UInt32Array`s for storing stat data for less memory noise
-- Manually runs GC for more accurate memory stats
-- Exposes lifecycle events for real-time data (`TODO`)
-- Saves results into a markdown file (with SVG graphs - `TODO`)
-- Shows basic stats and events in a terminal (`TODO`)
-- Runs in interactive mode with all data/events/graphs (`TODO`)
+- Waits until V8 optimizations kick in and benchmarks become less noisy
+- Gets rid of performance noise caused by benchmark wrapper functions 
+- Reuses a couple of `UInt32Array`s to store stats for less memory noise
+- Runs GC before every benchmark and suite for less memory noise 
+- Uses high resolution time in nanoseconds for more accurate cpu results 
+- Exposes lifecycle events listening to data in real-time 
+- Prints colorful benchmark results into a terminal
+- Allows combining different output strategies (terminal/markdown/etc.)
 
-## Example 
+## Installation
+
+```shell
+yarn add benchpress
+```
+
+## Example
 
 ```ts
-// you can pass options into the preset
+import { preset } from "benchpress";
+
+// define suite preset with options 
 const defaultSuite = preset();
 
 // define your suite with benchmarks
-const testBenchmark = defaultSuite({
-	emptyAsync: async () => { /* */ },
-	emptySync: () => { /* */ }
+const testBenchmark = defaultSuite("Test", {
+  emptyAsync: async () => {},
+  emptySync: () => {},
+});
+
+(async () => {
+	// collect data and print them into a terminal 
+	useTerminal();
+
+	// run all benchmarks and trigger events
+	await testBenchmark();
+})();
+```
+
+## API
+
+### `preset`
+
+Returns a `suite` preset with predefined options.
+
+```ts
+const suite = preset({
+	// options
 });
+```
 
-// run all benchmarks and log the results
-for await (const result of testBenchmark()) {
-	console.log(result);
+These are the default options:
+
+```ts
+{
+  cpu: {
+    chunkSize: 100,
+    compareSize: 10,
+    rangePercent: 10,
+  },
+  ram: {
+    chunkSize: 5,
+    compareSize: 5,
+    rangePercent: 5,
+  },
+  offset: {
+    allow: true,
+    rangePercent: 5,
+  },
+  gc: {
+    allow: true,
+  }
 }
 ```
 
-## `TODO` API
+### `suite` 
+
+Returns a function which asynchronously runs all provided benchmarks.
+
+```ts
+const runBenchmarks = suite("Name", {
+	// benchmarks	
+});
+```
+
+Since all suites share the same references to internal objects you should never run multiple suites at the same time (not awaiting them). This is how multiple suites should be run:
 
-## Notes
+```ts
+await firstSuite();
+await secondSuite();
+await thirdSuite();
+```
+
+### `useTerminal`
 
-`NOTE` I should probably create a section based on each one of these notes to describe things in more detail. 
+Listens to events and prints suite and benchmark results into a terminal.
 
-- Since we use nanoseconds for measuring how long each function takes to execute and there is `1_000_000_00` nanoseconds in a second then the most op/s a benchmark can get is `1_000_000_000` (in such a case it means the function took <0, 1> nanoseconds to execute)
-- Before any user defined benchmarks are run we run 4 hidden benchmarks (async-cpu, async-ram, sync-cpu, sync-ram) to determine the cost of the wrapper functions used for benchmark definitions and subtract those numbers from all of the user defined benchmarks (so if you benchmark an empty function it should give you 0ns and 0kbs since you're basically benchmarking nothing)
+```ts
+// subscribe to events
+useTerminal();
 
-## Questions
+// run suite which publishes data to the events
+await runBenchmarks();
+```
 
-`NOTE` These are currently just my notes so I don't forget.
+## Events
 
-- Should I pass a specific name to a suite or require an object with suites as values for all of the modes?
+The `suite` by itself doesn't return any data. For consuming suite and benchmarks data you should listen to events. All events are prefixed with `$`.
 
-## `TODO` Modes
+### `$suiteStart`
 
-`NOTE` These are currently just my notes so I don't forget.
+```ts
+{ 
+	suite: Name; 
+	benchmarks: string[]; 
+}
+```
+
+### `$suiteOffsets`
+
+```ts
+{
+  suite: Name;
+  offsets: Offsets;
+}
+```
 
-### Markdown mode
+### `$suiteEnd`
 
-- Outputs markdown into a file (folder has to exist beforehand)
-- Data should be in a table with all important data (currently just op/s and kbs)
-- If possible should generate SVG graphs and include them into the markdown file (`TODO`)
+```ts
+{ 
+	suite: Name;
+}
+```
 
-### `TODO` Static mode
+### `$benchmarkStart`
 
-- Shows real-time data but can't use keyboard
-- Doesn't show a graph, only the basic info
-- Should be clear what is happening at any given moment to let users know why they're waiting
+```ts
+{ 
+	suite: Name; 
+	benchmark: Name;
+}
+```
 
-### `TODO` Interactive mode
+### `$benchmarkEnd`
 
-- Shows real-time data from all benchmark iterations
-- It should have different "screen" per each benchmark
-- I can switch between different screens
-- I should be able to also show all results in one big graph
-- I should be able to restart a benchmark from the TUI
-- The restart data should be merged instead of using the latest (maybe??)
-- The cold part should have different color from the hot results
+```ts
+{
+  suite: Name;
+  benchmark: Name;
+  cpu: Offset;
+  ram: Offset;
+}
+```

examples/arrayLoops.ts

@@ -1,12 +1,12 @@
 import { preset } from "../src";
-import { modeMarkdown } from "../src/modes";
+import { useTerminal } from "../src/modes";
 
 const LENGTH = 1_000;
 const DATA = [...new Array(LENGTH)].map(() => Math.random() * 10);
 const RESULT = { _: 0 };
 
 const defaultSuite = preset();
-const emptyFunctions = defaultSuite({
+const emptyFunctions = defaultSuite("Array loops", {
   for: () => {
     for (let i = 0; i < LENGTH; ++i) {
       RESULT._ = DATA[i];
@@ -30,5 +30,7 @@ const emptyFunctions = defaultSuite({
 });
 
 (async () => {
-  await modeMarkdown(emptyFunctions, "./examples/arrayLoops.md");
+  useTerminal();
+
+  await emptyFunctions();
 })();

examples/emptyFunctions.ts

@@ -1,8 +1,8 @@
 import { preset } from "../src";
-import { modeMarkdown } from "../src/modes";
+import { useTerminal } from "../src/modes";
 
 const defaultSuite = preset();
-const emptyFunctions = defaultSuite({
+const emptyFunctions = defaultSuite("Empty functions", {
   emptyAsync: async () => {
     /* */
   },
@@ -12,5 +12,7 @@ const emptyFunctions = defaultSuite({
 });
 
 (async () => {
-  await modeMarkdown(emptyFunctions, "./examples/emptyFunctions.md");
+  useTerminal();
+
+  await emptyFunctions();
 })();

package.json

@@ -30,24 +30,29 @@
   "devDependencies": {
     "@release-it/keep-a-changelog": "3.1.0",
     "@swc/cli": "0.1.62",
-    "@swc/core": "1.3.35",
+    "@swc/core": "1.3.40",
     "@swc/helpers": "0.4.14",
-    "@types/jest": "29.4.0",
-    "@types/node": "18.13.0",
-    "@typescript-eslint/eslint-plugin": "5.52.0",
-    "@typescript-eslint/parser": "5.52.0",
+    "@types/jest": "29.4.1",
+    "@types/node": "18.15.3",
+    "@typescript-eslint/eslint-plugin": "5.55.0",
+    "@typescript-eslint/parser": "5.55.0",
     "auto-changelog": "2.4.0",
-    "esbuild": "0.17.8",
+    "chalk": "4.1.2",
+    "esbuild": "0.17.11",
     "esbuild-plugin-swc": "1.0.1",
-    "eslint": "8.34.0",
-    "eslint-config-prettier": "8.6.0",
+    "eslint": "8.36.0",
+    "eslint-config-prettier": "8.7.0",
     "husky": "8.0.3",
-    "jest": "29.4.3",
+    "jest": "29.5.0",
     "npm-dts": "1.3.12",
     "prettier": "2.8.4",
-    "release-it": "15.6.0",
+    "release-it": "15.8.0",
     "ts-jest": "29.0.5",
     "ts-node": "10.9.1",
     "typescript": "next"
+  },
+  "dependencies": {
+    "elfs": "0.0.7",
+    "ueve": "1.2.2"
   }
 }