Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use async with esplora-reqwest #115

Merged
merged 2 commits into from
Sep 2, 2022

Conversation

danielabrozzoni
Copy link
Member

@danielabrozzoni danielabrozzoni commented Aug 24, 2022

Description

We previously had the esplora-reqwest feature, but it would use
sync reqwest, as the "async-interface" feature in BDK wasn't set.
This commit sets this feature so that using esplora-reqwest always
uses async mode.

Notes to the reviewers

Checklists

All Submissions:

  • I've signed all my commits
  • I followed the contribution guidelines
  • I ran cargo fmt and cargo clippy before committing

New Features:

  • I've added tests for the new feature
  • I've added docs for the new feature
  • I've updated CHANGELOG.md

Copy link
Member

@afilini afilini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 2d8b1fa

@rajarshimaitra rajarshimaitra mentioned this pull request Aug 28, 2022
6 tasks
@rajarshimaitra
Copy link
Contributor

I am not sure if this is us or some problem with generic https calls with async-reqwest, but I am getting this error while trying out a wallet sync

$ ./target/debug/bdk-cli wallet -d "wpkh(tprv8ZgxMBicQKsPeuazF16EdPZw84eHj55AU8ZKgZgdhu3sXcHnFgjzskfDvZdTaAFHYNCbKqrurFo9onSaT7zGT1i3u3j7LKhVZF5sJA39WPN/86'/1'/0'/0/*)#40hv8z77" sync
[2022-08-29T10:12:21Z ERROR bdk_cli] Esplora(Reqwest(reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("blockstream.info")), port: None, path: "/testnet/api//scripthash/3149608bac07ddc9d628dd85fa0a275d88cc60b4efded5499230973557151e80/txs", query: None, fragment: None }, source: hyper::Error(Connect, "invalid URL, scheme is not http") }))

But ureq is working fine on the same esplora endpoint..

Cargo.toml Outdated
@@ -44,7 +45,8 @@ electrum = ["bdk/electrum"]
compact_filters = ["bdk/compact_filters"]
esplora = []
esplora-ureq = ["esplora", "bdk/use-esplora-ureq"]
esplora-reqwest = ["esplora", "bdk/use-esplora-reqwest"]
async-interface = ["bdk/async-interface"]
esplora-reqwest = ["esplora", "bdk/use-esplora-reqwest", "async-interface"]
Copy link
Contributor

@rajarshimaitra rajarshimaitra Aug 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah found the problem.. We need to also enable default-tls for reqwest..

Suggested change
esplora-reqwest = ["esplora", "bdk/use-esplora-reqwest", "async-interface"]
esplora-reqwest = ["esplora", "bdk/use-esplora-reqwest", "bdk/reqwest-default-tls", "async-interface"]

@notmandatory
Copy link
Member

if we're not going to revert #114 then this PR also needs to be rebased. :-)

maybe_async is used regardless of which feature is activated, so
it shouldn't be behind the cfg()
@danielabrozzoni
Copy link
Member Author

danielabrozzoni commented Aug 29, 2022

Rebased :)
@rajarshimaitra I'm ok with your change, but first, could you help me understand why the tests weren't failing? :( With which features did you compile?

@notmandatory
Copy link
Member

notmandatory commented Aug 29, 2022

@raj I'm ok with your change, but first, could you help me understand why the tests weren't failing? :( With which features did you compile?

This seems to be the issue if you use the esplora-reqwest features and try to connect to an https URL (which is default) then we get the below error:

% RUST_LOG=debug cargo run --features esplora-reqwest -- wallet --descriptor "wpkh(tpubEBr4i6yk5nf5DAaJpsi9N2pPYBeJ7fZ5Z9rmN4977iYLCGco1VyjB9tvvuvYtfZzjD5A8igzgw3HeWeeKFmanHYqksqZXYXGsw5zjnj7KM9/*)" sync
...
ERROR bdk_cli] Esplora(Reqwest(reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("blockstream.info")), port: None, path: "/testnet/api//scripthash/ca0948ef1c5bf5a023f22ed75275d7491c194476c11fbcfc63c0d25ed8234124/txs", query: None, fragment: None }, source: hyper::Error(Connect, "invalid URL, scheme is not http") }))

I confirmed that with @raj's suggestion this problem is fixed. We don't have any CI tests for this sort of thing so only found with the manual testing, not sure it's worth setting up integration tests for this project which is sort of meant to be used for manual testing (and other stuff).

@rajarshimaitra
Copy link
Contributor

We don't have any CI tests for this sort of thing so only found with the manual testing, not sure it's worth setting up integration tests for this project which is sort of meant to be used for manual testing (and other stuff).

Its definitely worth setting up one, and now that we have automated backend we can use that to cook up a very simple integration test framework using std library only, where we can test out basic operations. This is included in this commit 2302ff5 of #102 . Its still very basic but it works for Rpc and Electrum. Esplora is disabled for now because I was facing some issues with it..

@rajarshimaitra
Copy link
Contributor

@rajarshimaitra I'm ok with your change, but first, could you help me understand why the tests weren't failing? :( With which features did you compile?

Yes as @notmandatory noted, we don't have functional tests yet for bdk-cli. We only check for clap structures and some few other functions..

@danielabrozzoni
Copy link
Member Author

Thanks for the explanation, I just updated the PR 🙏🏻

Copy link
Member

@notmandatory notmandatory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 81e7226

Copy link
Contributor

@rajarshimaitra rajarshimaitra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tACK 81e7226

@notmandatory
Copy link
Member

@rajarshimaitra is it ok to merge this before #102? I don't think it will conflict much.

@rajarshimaitra
Copy link
Contributor

@rajarshimaitra is it ok to merge this before #102? I don't think it will conflict much.

Yes it is.. I will rebase #102 once this is in..

Cargo.toml Outdated
@@ -27,6 +27,7 @@ fd-lock = { version = "=3.0.2", optional = true }
regex = { version = "1", optional = true }
bdk-reserves = { version = "0.19", optional = true}
electrsd = { version= "0.12", features = ["trigger", "bitcoind_22_0"], optional = true}
tokio = { version = "1", features = ["rt", "macros", "rt-multi-thread"] }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be optional and only included with async-interface

Copy link
Member

@afilini afilini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 3ad5b02

We previously had the esplora-reqwest feature, but it would use
sync reqwest, as the "async-interface" feature in BDK wasn't set.
This commit sets this feature so that using `esplora-reqwest` always
uses async mode.
Copy link
Member

@afilini afilini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reACK 8d876ef

@afilini afilini merged commit a9d7726 into bitcoindevkit:master Sep 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants