What output modes should I add?

[yamiteru]

Currently we have these output modes:

• useTerminal - pretty terminal output
• useTerminalCompact - compact terminal output (similar to benchmark.js)

I was thinking about these:

• useTerminalTable - print results as a table into terminal
• useTerminalInteractive - interactive TUI with ASCII graphs
• useFileSvg - generate SVG (line?) graph
• useFileMarkdown - save benchmark results into a markdown file
• useWebsocket - send results over a websocket (useful for a web app which could visualize the captured data)

---

Which output mode do you want?
(also please specify how they should work/look like)

[H4ad]

I think an option of useTerminalJson() could be good for CI/CD.

Also, is a good thing to think of a way to compose those modes, like enabling an option to output the result to a file or directly to the terminal (the default), in this way, we can pretty print for the terminal in CI/CD and they save in a file the results that are easier to read.

You can use nodejs-bench-operations as an example, Rafael created it to benchmark nodejs operations and it uses CI/CD to automate the generation of the tables and other stuffs. Maybe you can study this repo to create APIs that could make the life of maintaining that repo way easier.

[yamiteru]

Thanks for the inspiration! Looks very interesting. I've been also thinking about ways to compose modes together and making them more modular. It's easy for simple stuff like JSON. You either save it or log it.

But for example the current useTerminal uses propcess.stdout for re-rendering the same line in terminal so making this one would be much more tricky.

I was thinking about using something Ink for creating the logs. In the case of saving a file I could just get the result of the render function and parse it and then save it. I think this would make things much easier.

Also I've been thinking about turning this into a monorepo and making the modes into standalone libraries. Theoretically they could be used with other benchmark utilities if provided with the same interfaces/types.

[H4ad]

Monorepo is generally a good idea, but for a benchmark library, I don't think it should be your first choice for extending functionality.

I want to install just one lib and be able to measure the performance of my code without having to install X, Y, Z.

However, using monorepo to control dependencies and in the end make the user install only one of the libraries, and have internal libraries just for code organization, I think it might be a good idea.

Perhaps you could combine the strategies, make it modular enough to be extended and allow for an ecosystem, and also bundle everything together in ìsitfast to be easier to install.

[H4ad]

Also, my suggestion to you is to prioritize the outputs based on public benchmarks repositories, like the repo from Rafael.

NodeJS also have some benchmark tooling, you can also read those files to learn more about how they are doing their benchmark tooling.

Ref: https://github.com/nodejs/node/tree/main/benchmark