Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: create add-dir benchmark #167

Merged
merged 11 commits into from
Jul 11, 2023
Merged

test: create add-dir benchmark #167

merged 11 commits into from
Jul 11, 2023

Conversation

SgtPooki
Copy link
Member

@SgtPooki SgtPooki commented Jun 28, 2023

Creates a new benchmark that compares adding directories with helia, js-ipfs, and kubo (via ipfsd-ctl)

Related issues

@SgtPooki SgtPooki requested a review from a team as a code owner June 28, 2023 20:42
@whizzzkid
Copy link
Contributor

@SgtPooki I created: #169

However both the baseline and fix benchmark numbers don't align on my machine, can we also create a benchmark runner github action to generate consistent benchmarks?

@SgtPooki
Copy link
Member Author

@SgtPooki I created: #169

However both the baseline and fix benchmark numbers don't align on my machine, can we also create a benchmark runner github action to generate consistent benchmarks?

what do you mean they don't align? They are different on your machine than mine? That's to be expected.

As far as a github action for running these benchmarks and keeping them posted somewhere.. I think that's an excellent idea and something we should create an issue for (see #170). It would be nice to output those benchmarks in line with the version of helia used when creating them so we can see how things change as we progress.

* fix: ⚡ 2x improvement

Signed-off-by: Nishant Arora <1895906+whizzzkid@users.noreply.github.com>

* fix: dag generation with parallelization at each level

---------

Signed-off-by: Nishant Arora <1895906+whizzzkid@users.noreply.github.com>
Co-authored-by: Russell Dempsey <1173416+SgtPooki@users.noreply.github.com>
@SgtPooki
Copy link
Member Author

SgtPooki commented Jul 3, 2023

ping @achingbrain

@achingbrain
Copy link
Member

@SgtPooki I made some changes to this PR in #171 - let me know what you think

Copy link
Contributor

@whizzzkid whizzzkid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

const parentDirectoryName = nodePath.dirname(dir).split(nodePath.sep).pop()
let rootCID = await unixFs.addDirectory({ path: parentDirectoryName }, unixFsAddOptions)

const children = await Promise.all(dirents.map(async dirent => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

* fix: use glob source for importing directories

- Use `globSource` from ipfs-utils for import
- Make all impls return the same CID
- Remove the dir size - if the CID is the same that's good enough

* chore: fix linting

* chore: remove benchmark from workspaces
@SgtPooki
Copy link
Member Author

@achingbrain are you good with merging this?

@SgtPooki SgtPooki merged commit afd569d into main Jul 11, 2023
17 checks passed
@SgtPooki SgtPooki deleted the benchmark/add-dir branch July 11, 2023 20:19
achingbrain added a commit that referenced this pull request Jan 8, 2024
Updates all deps and fixes linting errors.

Renames the `dht` routing to `libp2p` since it's used any and all configured libp2p Content Routers instead of the low-level DHT interface for several releases now.

BREAKING CHANGE: uses multiformats v13 and helia v3, renames `dht` routing to `libp2p`
achingbrain pushed a commit that referenced this pull request Jan 8, 2024
## [@helia/ipns-v4.0.0](https://github.com/ipfs/helia-ipns/compare/@helia/ipns-v3.0.1...@helia/ipns-v4.0.0) (2024-01-08)

### ⚠ BREAKING CHANGES

* uses multiformats v13 and helia v3, renames `dht` routing to `libp2p`

### Features

* update helia to v3 and multiformats to v13 ([#167](ipfs/helia-ipns#167)) ([a0381b9](ipfs/helia-ipns@a0381b9))

### Bug Fixes

* make @libp2p/interface a dependency ([#159](ipfs/helia-ipns#159)) ([546ecf0](ipfs/helia-ipns@546ecf0)), closes [#158](ipfs/helia-ipns#158)
@achingbrain achingbrain mentioned this pull request Jan 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

3 participants