Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hide library that is only used by WASM builds behind a feature #157

Merged
merged 4 commits into from
Dec 16, 2022

Conversation

MitchTurner
Copy link
Contributor

@MitchTurner MitchTurner commented Dec 7, 2022

CML was causing some weird cyclical dependencies when used with other libraries that depended on getrandom:

stdout :     Updating crates.io index
    Updating git repository `https://github.com/txpipe/aiken.git`
error: cyclic package dependency: package `getrandom v0.2.8` depends on itself. Cycle:
package `getrandom v0.2.8`
    ... which satisfies dependency `getrandom = "^0.2.3"` of package `ahash v0.7.6`
    ... which satisfies dependency `ahash = "^0.7.0"` of package `hashbrown v0.12.3`
    ... which satisfies dependency `hashbrown = "^0.12"` (locked to 0.12.3) of package `indexmap v1.9.2`
    ... which satisfies dependency `indexmap = "^1.5.2"` (locked to 1.9.2) of package `serde_json v1.0.89`
    ... which satisfies dependency `serde_json = "^1.0"` of package `wasm-bindgen v0.2.82`
    ... which satisfies dependency `wasm-bindgen = "^0.2.82"` of package `js-sys v0.3.59`
    ... which satisfies dependency `js-sys = "^0.3"` of package `getrandom v0.2.8`
    ... which satisfies dependency `getrandom = "^0.2.0"` of package `const-random-macro v0.1.15`
    ... which satisfies dependency `const-random-macro = "^0.1.15"` of package `const-random v0.1.15`
    ... which satisfies dependency `const-random = "^0.1.6"` of package `ahash v0.3.8`

Removing this getrandom from CML seems to fix it and CML still compiles.

Update: This solution doesn't work because it breaks WASM builds. Instead I've moved the getrandom to the wasm deps and put it behind a wasm feature. This required me to add that feature to the WASM build scripts as well.

@MitchTurner
Copy link
Contributor Author

Should I bump the version?

Copy link
Contributor

@rooooooooob rooooooooob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's used in WASM builds which fail if you remove it. If we're going to be excluding it to get around issues in other dependencies, we should move it to only WASM builds instead.

@MitchTurner
Copy link
Contributor Author

MitchTurner commented Dec 7, 2022

It's used in WASM builds which fail if you remove it. If we're going to be excluding it to get around issues in other dependencies, we should move it to only WASM builds instead.

Adding it to

[target.'cfg(all(target_arch = "wasm32", not(target_os = "emscripten")))'.dependencies]

Caused my code to break again. Maybe we could put it behind a feature or something? Not sure what the best approach is here, I'm not super familiar with WASM builds.

@MitchTurner
Copy link
Contributor Author

@rooooooooob I'm not really sure how to move forward on this. As I mentioned above, just adding it to the Wasm deps gave me an error:

stdout :     Updating crates.io index
error: failed to select a version for `getrandom`.
    ... required by package `uuid v1.2.2`
    ... which satisfies dependency `uuid = "^1.1.2"` (locked to 1.2.2) of package `naumachia v0.2.0 (/home/mitchell/personal/naumachia)`
    ... which satisfies path dependency `naumachia` (locked to 0.2.0) of package `always-succeeds-contract v0.2.0 (/home/mitchell/personal/naumachia/sample-dApps/always-succeeds-contract)`
versions that meet the requirements `^0.2` (locked to 0.2.7) are: 0.2.7

all possible versions conflict with previously selected packages.

  previously selected package `getrandom v0.2.8`
    ... which satisfies dependency `getrandom = "^0.2.8"` of package `cardano-multiplatform-lib v3.1.1 (/home/mitchell/personal/forks/cardano-multiplatform-lib/rust)`
    ... which satisfies dependency `cardano-multiplatform-lib = "^3.1.1"` (locked to 3.1.1) of package `blockfrost-http-client v0.0.8 (/home/mitchell/personal/blockfrost-http-client)`
    ... which satisfies dependency `blockfrost-http-client = "^0.0.8"` (locked to 0.0.8) of package `naumachia v0.2.0 (/home/mitchell/personal/naumachia)`
    ... which satisfies path dependency `naumachia` (locked to 0.2.0) of package `always-succeeds-contract v0.2.0 (/home/mitchell/personal/naumachia/sample-dApps/always-succeeds-contract)`

failed to select a version for `getrandom` which could resolve this conflict

A slightly different error from before, but still not happy.

If you want to repro this, btw, you can cargo new an empty project and include

[dependencies]
aiken-project = { git = "https://github.com/txpipe/aiken.git", branch = "main"}
cardano-multiplatform-lib = "3.1.1"

in the cargo.toml.

Do you have any suggestions?

@rooooooooob
Copy link
Contributor

If you want to repro this, btw, you can cargo new an empty project and include

Even if I do this without the CML dependency (Just the Aiken one) I get:

error: failed to select a version for the requirement minicbor = "^0.18"
candidate versions found which didn't match: 0.17.1, 0.17.0, 0.16.1, ...
location searched: crates.io index
required by package pallas-codec v0.14.0

despite it existing on crates.io as well as if I locally do cargo search minicbor that 1.18.0 shows up. I tried deleting .cargo/registry and such since I figured it was a cache issue and that's what searching suggested to do but it didn't help.

I'm not sure what that is about, but from what you posted it sounds like moving it to WASM likely fixed the original issue, but now we have a different one with dependency conflicts. I can't even cargo tree the aiken repo since I get that same issue as above, but judging by your error message it's due to naumachia which luckily you are the author of. It's using an older version of CML and recently we bumped getrandom in #146 so it could be because you're (indirectly) using two different versions of CML which in turn use different version of getrandom. Could you try bumping the version of CML used in naumachia?

@MitchTurner
Copy link
Contributor Author

Very strange that you're having issues with minicbor. I have that MRE up on my computer right now and it's not saying anything about minicbor. Yours is 2021 and a bin?

@MitchTurner
Copy link
Contributor Author

MitchTurner commented Dec 8, 2022

Yeah, it's mentioning Naumachia because that's where I'm running the code right now. I get the exact same error in my other library, just caused by a different dep. So, in my above error code I'm pretty sure CML is colliding with uuid and in my other library it's calling out rand_core.

(I can't repro the above case so I struck it out)

Just for fun, I cut down the getrandom down to 0.2.7(keeping it under the WASM deps), but that caused the cyclical error to return :(

@MitchTurner
Copy link
Contributor Author

Okay. There is the option of pinning getrandom to 0.2.8 in my library (similar to what y'all are doing), which seems to get rid of the second error. But the first error returns in that case.

@MitchTurner
Copy link
Contributor Author

@rooooooooob
I think I solved it by putting getrandom behind a wasm feature. I then added that feature to all the wasm npm build scripts.

@MitchTurner
Copy link
Contributor Author

Unrelated to my changes, there are a bunch of warnings from the rust compiler that are cluttering my compiles. I might clean those up as well in my PR, if that's okay.

I could do it separately too. I'd like to get these changes through.

@MitchTurner MitchTurner changed the title Remove unused library to make compiler happy Hide library that is only used by WASM builds behind a feature Dec 8, 2022
@@ -1,1161 +1,8 @@
{
"name": "cardano-multiplatform-lib",
"version": "3.1.1",
"lockfileVersion": 2,
"lockfileVersion": 1,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SebastienGllmt Do you remember anything about this? I vaguely recall at some point it going from version 1 to 2 and adding a bunch of lines (like the ones removed here).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depending on which version of NPM you have installed, it will change the lockfileVersion. It's annoying because if you have multiple people on your team with different NPM versions, it causes issues. Maybe v2 has been out long enough that it's less of an issue now, but still annoying.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

> npm -v
8.5.4

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this not going to work? Do I need to change something?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you need to undo your changes to package-lock.json and then run npm install again making sure you're on a recent enough version that supports the v2 format (8.5.4 should be recent enough)

Copy link
Contributor Author

@MitchTurner MitchTurner Dec 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kk. Ran that. I'm confused why it acted differently this time. I didn't change my npm version. Maybe it was another script that messed it up before, like rust:build-nodejs maybe.

@MitchTurner
Copy link
Contributor Author

@rooooooooob @SebastienGllmt any other changes?

Copy link
Contributor

@SebastienGllmt SebastienGllmt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but @rooooooooob needs to sign off on this

Copy link
Contributor

@rooooooooob rooooooooob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM on my end too.

@rooooooooob rooooooooob merged commit 6853544 into dcSpark:develop Dec 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants