feat: use nwaku instead of go-waku #87

richard-ramos · 2024-02-08T15:48:09Z

This PR / branch will contain all the changes related to using nwaku instead of go-waku.
Once we achieve feature parity with current master, we need to reset that branch to point to the latest commit of this PR.

Compile nwaku and generate bindings automagically
Replace function calls to invoke nwaku's libwaku.h
Create / Start / Stop node
Relay Publish / Subscribe
Event callback
Result + Error handling

In separate PRs, the following functionality needs to be implemented

Implement suggestions from Nwaku 2 event callback test #88

Use Opaque type like in

waku-rust-bindings/waku-bindings/src/node/mod.rs

Lines 26 to 34 in 5abd9cc

    
           pub struct OpaqueWakunodePtr { 
        
               obj_ptr: *mut c_void, 
        
           } 
        
           unsafe impl Send for OpaqueWakunodePtr {} 
        
           /// Handle to the underliying waku node 
        
           pub struct WakuNodeHandle { 
        
               ctx: Arc<Mutex<OpaqueWakunodePtr>>,

Destroy node
Add example program and docs
Fix TODOs
Expose any additional function

richard-ramos · 2024-02-08T18:26:38Z

This PR so far compiles waku-sys succesfully generating binding.rs

cd waku-sys
cargo build

The first time does take a while because it's building nim and downloading the submodules, so maybe we can consider generating .so/.a for different architectures and uploading them to the releases, if build times become a problem.

richard-ramos · 2024-02-13T20:21:38Z

waku-bindings now compiles succesfully and runs some basic tests

cd waku-bindings
cargo test

I added rln as a dependency so nwaku can run succesfully (it seems to no longer be an optional dependency). Also linked some other libraries and had to compile a libbacktrace_wrapper.o fike to link it succesfully. This has been tested on a linux machine. We should try on other architectures.

richard-ramos · 2024-02-14T21:09:17Z

We can now invoke the relay functions to publish and subscribe. I've run into some trouble setting up the event handler, which seems to work, but in the tests it fails when we attempt to push a value to a tokio channel (fails on line 48).

waku-rust-bindings/waku-bindings/tests/node.rs

Lines 45 to 53 in 24d7dd8

    
           println!("==============="); 
        
           println!("ID: {}", id); 
        
           println!("Sending to channel...."); 
        
           futures::executor::block_on(tx.send(Response { 
        
               id: id.to_string(), 
        
               payload, 
        
           })) 
        
           .expect("send response to the receiver"); 
        
           println!("Sent!");

I'm investigating what's up with this.

richard-ramos · 2024-02-16T14:57:36Z

Thanks to @danielSanchezQ 's help, the event callback now works!

cd waku-bindings
cargo test default_echo -- --nocapture --exact

I'll now proceed to improve error handling as well as including suggestions from #88

richard-ramos · 2024-02-20T20:08:48Z

This PR is mostly complete.
Once waku-org/nwaku#2461 is merged, I'll fix the waku_flow test to use a rust node instead of using the fleet. I will not add more code to the PR, since it will get too complicated to review. Missing functionality will be done on a separate PR.

richard-ramos · 2024-02-21T20:06:36Z

@danielSanchezQ @Ivansete-status @SionoiS
I've marked this PR as ready for review. It contains a lot of changes (sorry!), but most of them are just deleting code or changing function signatures, with just a small section dedicated to actually use nwaku instead of go-waku

@danielSanchezQ , don't worry too much about all the deleted code. I'm going to add it back in future PRs towards this branch once the functionality is available in nwaku c-bindings :)

SionoiS

Nice base to build upon!

waku-bindings/src/node/events.rs

waku-bindings/src/node/management.rs

SionoiS · 2024-02-21T20:42:33Z

waku-bindings/src/node/relay.rs

    message: &WakuMessage,
-    pubsub_topic: Option<WakuPubSubTopic>,
+    pubsub_topic: &String,


In the future we may want to do something like pubsub_topic: impl Into<WakuPubsubTopic> and impl From<String> for WakuPubsubTopic Or even better remove pubsub topic and use shards with the appropriate type constraints.

Ivansete-status

Wonderful! Thanks for it!
I just added some minor comments

Ivansete-status · 2024-02-26T12:56:47Z

waku-bindings/src/node/management.rs

-pub fn waku_peer_id() -> Result<PeerId> {
+/// nwaku version
+#[allow(clippy::not_unsafe_ptr_arg_deref)]
+pub fn waku_version(ctx: *mut c_void) -> Result<String> {


I think we can use const in this particular case. Maybe we could check other fns and use the most restrictive option in each case.

Suggested change

pub fn waku_version(ctx: *mut c_void) -> Result<String> {

pub fn waku_version(ctx: *const c_void) -> Result<String> {

Ah! in this case we're using bindgen to generate the bindings for libwaku.h automagically in https://github.com/waku-org/waku-rust-bindings/blob/nwaku/waku-sys/build.rs#L68-L79 and it translates *void to *mut c_void.
I could make the change and then cast the variable from *const to *mut right before interacting with nwaku, but since these functions are not exposed to the end user. We expose a friendlier API:

waku-rust-bindings/waku-bindings/src/node/mod.rs

Lines 48 to 50 in 646f6f0

pub fn version(&self) -> Result<String> {

management::waku_version(&self.ctx)

}

)

I'm thinking we could accept to use a *mut in this case, just to not have to do the cast.

Ivansete-status · 2024-02-26T12:58:53Z

waku-bindings/src/node/mod.rs

    /// Start a Waku node mounting all the protocols that were enabled during the Waku node instantiation.
    /// as per the [specification](https://rfc.vac.dev/spec/36/#extern-char-waku_start)
-    pub fn start(self) -> Result<WakuNodeHandle<Running>> {
-        management::waku_start().map(|_| WakuNodeHandle(Default::default()))
+    pub fn start(&self) -> Result<()> {


In a future PR we might need to add error handling ( it is actually pending to be implemented in libwaku.nim too .)

… return code (#97) * refactor: handle return code and use an enum to handle responses * fix: nwaku does not return an envelope hash on publish

* refactor: node handle constructor and messageId on publish * refactor: add back typestate * chore: rename messageId to messageHash

* update nwaku vendor to v0.33.1 * build.rs: add negentropy dependency and cmdCount cmdLine dependencies * fix: call waku_setup when instantiating waku_new * Properly decode a Vec<Multiaddr> * First commit tic-tac-toe * adding some simple game logic to coordinate the turns a little * some logic to panic if a proper event callback hasn't been set * restoring back the type state pattern introduced by Richard * new PubsubTopic type * fix clippy issues --------- Co-authored-by: Richard Ramos <info@richardramos.me>

github-actions bot assigned richard-ramos Feb 8, 2024

feat: use nwaku instead of go-waku

9750a32

richard-ramos force-pushed the nwaku branch from d1a900d to 9750a32 Compare February 8, 2024 15:55

richard-ramos added 5 commits February 8, 2024 17:15

chore: remove unneeded files (for now)

9d73230

refactor: remove unneeded functions and change some data types

8755d9a

chore: remove code no longer needed (for now)

1d8626b

chore: link required libraries and RLN

ca72e70

chore: make sure waku_new, waku_start and waku_stop work

5687e25

fix: event callback signature

1f9283a

richard-ramos force-pushed the nwaku branch from ba3bb13 to 24d7dd8 Compare February 14, 2024 21:06

fix: event handler

685a6ae

richard-ramos force-pushed the nwaku branch from 24d7dd8 to 685a6ae Compare February 16, 2024 14:45

richard-ramos mentioned this pull request Feb 16, 2024

fix(bindings): base64 payload and key for content topic waku-org/nwaku#2435

Merged

4 tasks

feat: expose waku_version

9c696e6

richard-ramos force-pushed the nwaku branch from 3a84e65 to 9c696e6 Compare February 20, 2024 20:06

richard-ramos added 3 commits February 21, 2024 11:59

feat: add listen addresses

79b8428

chore: use rust node address instead of fleet node in tests

ac96b83

chore: remove example (for now)

9d3b575

richard-ramos marked this pull request as ready for review February 21, 2024 19:58

richard-ramos requested review from danielSanchezQ, SionoiS and Ivansete-status February 21, 2024 20:06

SionoiS approved these changes Feb 21, 2024

View reviewed changes

danielSanchezQ approved these changes Feb 22, 2024

View reviewed changes

refactor: remove getters

c434e9e

Ivansete-status approved these changes Feb 26, 2024

View reviewed changes

richard-ramos added 4 commits February 26, 2024 11:13

refactor: hide *mut c_void behind WakunodeContext (#96)

646f6f0

refactor: use a enum for handling the responses, and don't ignore the…

a10a5c2

… return code (#97) * refactor: handle return code and use an enum to handle responses * fix: nwaku does not return an envelope hash on publish

refactor: node handle constructor and messageHash on publish (#98)

69a4872

* refactor: node handle constructor and messageId on publish * refactor: add back typestate * chore: rename messageId to messageHash

feat: example app (#99)

201a38a

richard-ramos linked an issue Mar 20, 2024 that may be closed by this pull request

chore: pack nwaku instead of go-waku #90

Open

richard-ramos and others added 3 commits March 21, 2024 09:17

chore: disable windows builds temporarily (#100)

1ed7dd4

feat: RLN Relay (#101)

7a2e4d1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: use nwaku instead of go-waku #87

feat: use nwaku instead of go-waku #87

richard-ramos commented Feb 8, 2024 •

edited

Loading

richard-ramos commented Feb 8, 2024

richard-ramos commented Feb 13, 2024 •

edited

Loading

richard-ramos commented Feb 14, 2024

richard-ramos commented Feb 16, 2024

richard-ramos commented Feb 20, 2024 •

edited

Loading

richard-ramos commented Feb 21, 2024

SionoiS left a comment

SionoiS Feb 21, 2024

Ivansete-status left a comment

Ivansete-status Feb 26, 2024

richard-ramos Feb 26, 2024

Ivansete-status Feb 26, 2024

	pub struct OpaqueWakunodePtr {
	obj_ptr: *mut c_void,
	}

	unsafe impl Send for OpaqueWakunodePtr {}

	/// Handle to the underliying waku node
	pub struct WakuNodeHandle {
	ctx: Arc<Mutex<OpaqueWakunodePtr>>,

	pub fn waku_version(ctx: *mut c_void) -> Result<String> {
	pub fn waku_version(ctx: *const c_void) -> Result<String> {

	pub fn version(&self) -> Result<String> {
	management::waku_version(&self.ctx)
	}

feat: use nwaku instead of go-waku #87

Are you sure you want to change the base?

feat: use nwaku instead of go-waku #87

Conversation

richard-ramos commented Feb 8, 2024 • edited Loading

richard-ramos commented Feb 8, 2024

richard-ramos commented Feb 13, 2024 • edited Loading

richard-ramos commented Feb 14, 2024

richard-ramos commented Feb 16, 2024

richard-ramos commented Feb 20, 2024 • edited Loading

richard-ramos commented Feb 21, 2024

SionoiS left a comment

Choose a reason for hiding this comment

SionoiS Feb 21, 2024

Choose a reason for hiding this comment

Ivansete-status left a comment

Choose a reason for hiding this comment

Ivansete-status Feb 26, 2024

Choose a reason for hiding this comment

richard-ramos Feb 26, 2024

Choose a reason for hiding this comment

Ivansete-status Feb 26, 2024

Choose a reason for hiding this comment

richard-ramos commented Feb 8, 2024 •

edited

Loading

richard-ramos commented Feb 13, 2024 •

edited

Loading

richard-ramos commented Feb 20, 2024 •

edited

Loading