How to create a effective Shared State that retriggers Widget building?

[Vollbrecht]

Hi and thank you for your great tool.

I played around with a couple of options to archive the above but i am still struggling on the Dart site of things. First here is roughly what i want to archive and how what i tried to archive that.

• Have most / all states only in Rust.
• The top level state is available to everybody, by shared references, and is passed around in Dart to every place where it is needed.
• There is a never ending async rust task that got access to the shared ref, and is updating state.
• There are widgets that are writing the state like buttons on press and so forth.
• They are dart widgets that read the state and should be updated whenever, either the long running async task or some other dart widget is updating the state.

My rust code would look roughly like.

pub struct RustState {
   items: Arc<RwLock<Vec<Item>>>,
}

impl RustState {
   #[frb(sync)]
   pub fn new() -> Self {
       Self {
           items: Arc::new(RwLock::new(Vec::new())),
       }
   }
   
     #[frb(sync)]
   pub fn get_items(&self) -> Vec<Item> {
       ////....
   }
   
    #[frb(sync)]
   pub fn remove_item(&self, item: Item) -{
       ////....
   }
   
   pub async fn long_running_async_method(&self, sink: StreamSink<i32> {
     /// This is waiting for internal library events and if they happen - update state, and call sink add() so it retriggers Dart code.
   }    


}

The simplest and most ergonomic way for me currently, in the case of the rust async task to also use a StreamSink parallel to the shared ref. Though i don't actually are using the StreamSink data, its only a hack to inform dart about updates so it rebuild widget state.

Now to the dart site where i am struggling currently most. I tried different approaches like writing custom ChangeNotifier classes or setState calls with lots of wrapper classes. My goal is still -> update a widget that had read the state through a shared ref it was build to retrigger when another dart widget changes. And if i could i want to avoid writing 200-300 lines of state wrappers in dart.

So i tried to readup on helpers that are provided by the bridge but i don't seam to fully grasp them. First i tried to use the #[frb(ui_state)] and using #[frb(ui_mutate)] on methods that would modify the state. Though that did not work out for me, because if a method that is annotated with ui_mutate is just a &self than it seams to not doing anything. And i cant mark it as &mut self, because at the same time lots of other places are holding a &self like the long running async task. In my testing it doesn't chrash but it just blocks the ui blockingly forever on that calls.

The generated code annotated with ui_state seams to provide something that it will autogenerate ui_notifiers for me. If that work's that would be really helpful though not sure if that is working only on &self

The only other way currently for dart to dart updates with a rust state is using callbacks, Though that seams also to not scale well, e.g if 5 widgets are reading state A, than i need 5 callbacks collected to notify them about changes.

I am totally open to scratch what i am doing if someone has a better idea how i should structure my app here.

[fzyzcjy]

Hi you are welcome!

Firstly, this may be possibly related: https://cjycode.com/posts/rust-ui-flutter/. Does the demo there help?

Secondly, also cc @patmuk who has tried more complex state-related design patterns.

[Vollbrecht]

Thanks for your answer!

I tried to use the pattern described in the block post, though it seams that i hold it wrong or dont fully understand whats going behind the scenes here.

when for example having this rust snippet;

	#[frb(ui_mutation)]
    pub fn increment(&mut self) {
	    self.count += 1;
    }

i have the problem that this argument takes a &mut self so i can only call it from dart / rust if i actually have exclusive access to RustState right?

Though as i mention i plan to have a global shared reference and inner mutable state proteced by Arc<RwLocks<> or your AutoOpaque impls.

For me it would be ok if i can write something like

	#[frb(ui_mutation)]
    pub fn setItem(&self, Item: item) {
	    self.item.write_blocking().push(item)
    }
    
    pub fn getInnerState(&self) -> Item {
        self.item.clone()
    }

and every-time i call increment() / setItem() i would get notified everywhere i am using getInnerState() such that a Widget would automatically be rebuild.

That doesn't seam the case with what is currently generated by ui_mutation or most likely i am holding it wrong!

what i am currently doing in the dart site of things is creating a Wrapper class that i call from withing dart. That would help me with dart to dart calls but not with rust to dart calls.

What i am doing looks roughly like this

class ItemModel with ChangeNotifier {
  final MyState state;

  ItemModel({required this.state});

  void setItem(Item item) {
    state.setItem(item: item );
    notifyListeners();
  }
  
}

and then on my Widget that i want to use it using ListenableBuilders. That way at least from dart calls i get updates. Though i thought with the frb ui_state some similar code is autogenerated, so i don't need to write it manually?

Another point is if i am now changing the inner State from withing rust itself i dont go through the ItemModel class and subsequently doesn't get notified. For the one place i am needing it i am using a StreamSink currently as i outlined in the beginning.

Every tip on how to structure my code better so i can mutate state from rust and dart at the same time and update it when i read it in dart would be appreciated !

[fzyzcjy]

I got ill, but on recovery now :)

Sad to hear that :( Hope you recover quickly!

[fzyzcjy]

Looks like @patmuk had some very detailed suggestions and discussions, so I will not read and think about it (too busy now for such a long discussion), but feel free to ping me if there is anything that needs me to discuss further!

[patmuk]

@fzyzcjy thanks! I feel much better today.
Yes, feel free to ignore - unless somebody mentions you directly. I will do my best to answer & update with my latest realizations :)

[fzyzcjy]

@patmuk Happy to hear you are getting much better!

Sure, ping me when needed :)

[patmuk]

@Vollbrecht to come back about the ChangeNotifier: This would need much more code to make work - one needs to trigger notifyListeners(), there is no ValueListenableBuilder, so the whole Widget needs to inherit from ChangeListener..., etc.

Using ValueNotifierwith ValueListenableBuilder needs much less code and is much more fine-grain/performant/

[patmuk]

Hi @fzyzcjy, (and hi @Vollbrecht)

I wonder if one can avoid .clone() when communicating state changes between Rust and Dart.
I tried out different approaches in a branch my sample project but did not find a way to do so.

I could not get your approach (frb_ui annotations) working and did not understood it from the code examples - but I assume that it misses thread safety yet and I saw that the state instance is created in the Dart side. Looking forward to your write-up and help improving it!

I create the state singleton in the Rust side - as Rust->Dart is zero-copy, the other way around not. As the business logic is in Rust, I think it is better to have the state singleton "living" there.

However, my "problem" is the following: I will handle photos and videos, so .clone()-ing them will be a performance issue. But most probably the solution will be "keep the app state small and use references for the payload" :)
Nevertheless, before going/continuing the clone() way, I would appreciate your feedback on it, if you see a way I did not. And if some approaches are anyways .clone() under the hood.

And, most of all, if RwLock can be handled.

But let me start with describing my approach:

• the app_state lives in a singleton on the rust side (lifecycle::APP_STATE).
• This is a global variable (static). As it is mutable, I need to wrap it in LazyLock.
• Arc doesn't work, as I wouldn't be able to mutate the state
• To handle concurrent access, the App_State is wrapped in an RwLock, so it is LazyLock<RwLock<App_State>>.
• My Rust code is triggered by Dart, which theoretically can call rust's functions concurrently (usual way frb works :) )

The Problem:

• When the state is changed (e.g. by add_todo("my new todo") I want to return it to Dart

• But I can't move out the reference from the RwLock (or any other struct for that matter), so
  return & APP_STATE.read().model.todo_items doesn't work

• here I am using clone, which I want to avoid

• I tried to return the RwLock, planing that I can call .read() on it from Dart.

• for that I wrapped it in an struct AppStateLock { lock: &RwLock<AppState> }

• ... but that did not work - RwLock keeps being opaque - I tried to add the crate (parkinglot) to the frb-config, but that did not work either.

So, probably Locks cannot be translated to Dart in a working way? But then again, isn't RustOpaque a Arc<Mutex>?
I considered writing a custom encode/decoder - but I did not figure out how to decode to Rust without creating a new RwLock instance.

Many thanks for reading so far, and please excuse my walll-of-text. I hope it was understandable.
What do you (or anyone else having read that) think? Would it make sense to build-in RwLock support on the Dart side for frb? Would this even be possible?
Or is .clone()-ing good enough, with references wrapped in a proxy struct?

[fzyzcjy]

Ok let me reply this super long post ;)

And want to display one item in Dart to the user

If it is just u32, then maybe
EDIT: I see your 1st post and just ignore this case

pub struct AppState {
   items: Vec<u32>
}
impl AppState {
    pub fn get_first_item(&self) -> u32 {
        self.items[0]
    }
}

If it is something very heavy, then maybe

pub struct AppState {
   items: Vec<RustAutoOpaque<Heavy>>
}
impl AppState {
    pub fn get_first_item(&self) -> RustAutoOpaque<Heavy> {
        self.items[0].clone() // only clones the Arc so it is lightweight
    }
}

Btw if pub items does not work because of the auto-generated accessors you can remove pub and have a try.

I am a bit tired now and may miss something. feel free to correct me if I am wrong!

Having something like fn f(&self) -> &Heavy may have some intrinsic difficulty: Suppose our State have 5 items and f borrows the first one. Then, only after you release the borrowed one in Dart (explicitly or implicitly by gc), you can touch the AppState mutably. This is something vanilla Rust requires: You cannot borrow your AppState both mutably and immutably. That's why I would suggest RustAutoOpaque (indeed Arc) to share ownership.

[patmuk]

Thanks - this looks promising!
I implemented a minimal sample, in app_core/src/application/state_test.rs.

Are you sure that only the Arc is cloned?
Here is the code:

use flutter_rust_bridge::frb;

use super::bridge::frb_generated::RustAutoOpaque;

pub struct AppState {
    pub items: Vec<RustAutoOpaque<Heavy>>,
}
impl AppState {
    pub fn get_first_item(&self) -> RustAutoOpaque<Heavy> {
        self.items[0].clone() // only clones the Arc so it is lightweight
    }
}

#[frb(opaque)]
#[derive(Debug, Clone)] // TODO remove clone
pub struct Heavy {
    pub heavy: u32,
}

If I don't annotate the struct Heavy with #[derive(Clone)] I get

error[E0599]: the method `clone` exists for struct `RustAutoOpaqueBase<Heavy, MoiArc<RustAutoOpaqueInner<Heavy>>>`, but its trait bounds were not satisfied
  --> app_core/src/application/state_test.rs:10:23
   |
10 |         self.items[0].clone() // only clones the Arc so it is lightweight
   |                       ^^^^^ method cannot be called due to unsatisfied trait bounds
...
14 | #[frb(opaque)]
   | -------------- doesn't satisfy `state_test::Heavy: Clone`
   |
  ::: /Users/patmuk/.cargo/registry/src/index.crates.io-6f17d22bba15001f/flutter_rust_bridge-2.3.0/src/rust_auto_opaque/mod.rs:13:1
   |
13 | pub struct RustAutoOpaqueBase<T: 'static, A: BaseArc<inner::RustAutoOpaqueInner<T>>>(
   | ------------------------------------------------------------------------------------ doesn't satisfy `_: Clone`
   |
   = note: the following trait bounds were not satisfied:
           `state_test::Heavy: Clone`
           which is required by `RustAutoOpaqueBase<state_test::Heavy, frb_generated::MoiArc<RustAutoOpaqueInner<state_test::Heavy>>>: Clone`
help: consider annotating `state_test::Heavy` with `#[derive(Clone)]`

Otherwise, using it from rust looks good:

fn add_item(appstate: &mut AppState) {
    appstate
        .items
        .push(RustAutoOpaque::new(Heavy { heavy: 10 }));
}
async fn display_first_item(appstate: &AppState) {
    let fist_item = appstate.get_first_item();
    let heavy = fist_item.read().await;
    print!("The first item is {:?}", heavy);
}

:)

Having something like fn f(&self) -> &Heavy may have some intrinsic difficulty (...)

Yes - this is where we would need a Lock. In my code I used an RwLock on the rust side and .clone() for returning values to Dart. And hence the idea to get rid of .clone() and return values by exposing a Lock on the Dart side as well. So it would be fn get(&Self) -> &Lock<Heavy>.
I will try if RustAutoOpaque works like a lock for this case. Looks good so far, only that the need for #[derive(Clone)] on Heavy looks suspicious.

[fzyzcjy]

Are you sure that only the Arc is cloned?

Hmm... I guess it is either because the content is really cloned (which does not look good), or because of Rust derive macro's limitation (which can be easily fixed and is indeed not a issue). Maybe we can try a simple example: For your Heavy struct, manually impl Clone for Heavy and print a log (or just panic) when the Heavy.clone is called. Then clone the RustAutoOpaque to see what happens.
If it is the latter (Rust derive's limitation), feel free to PR to change #[derive(Clone)] struct RustAutoOpaque into a manually implemented Clone to avoid this.
From my memory, years ago when RustOpaque was implemented there was something like this as well and it seemed to be the latter.

[patmuk]

@fzyzcjy thanks, will do so! Unfortunately my setup crashed, and I have no clue why ... filed an issue: #2271

[fzyzcjy]

You are welcome! Replied to that issue just now

[patmuk]

@fzyzcjy and @Vollbrecht ,
it looks like everything works, without using clone!
Here is the code.
I minimized the example and removed all unneeded files.

On the rust side there is only state.rs.
The important part of that code is:

use flutter_rust_bridge::frb;
use super::bridge::frb_generated::RustAutoOpaque;

#[derive(Default)]
pub struct AppState {
    pub items: Vec<RustAutoOpaque<Heavy>>,
}
impl AppState {
    pub fn new() -> Self {
        Self {
            items: vec![RustAutoOpaque::new(Heavy { heavy: 42 })],
        }
    }
    pub fn get_last_item(&self) -> Option<RustAutoOpaque<Heavy>> {
        self.items.last().cloned()
        // .expect("there should be at least one item")
        // .clone() // only clones the Arc so it is lightweight
    }
    pub fn add_item(&mut self, item: u32) {
        self.items.push(RustAutoOpaque::new(Heavy { heavy: item }));
    }
    pub fn remove_item(&mut self, index: u32) {
        self.items.remove(index as usize);
    }
}

and the Heavy content:

#[frb(opaque)]
#[derive(Debug)] // TODO remove clone
pub struct Heavy {
    pub heavy: u32,
}

impl Clone for Heavy {
    fn clone(&self) -> Self {
        panic!("I should not be called");
        // Self { heavy: self.heavy.clone() }
    }
}

Clone has to be implemented, otherwise the return types do not work. But it is not called - assumedly Arc.clone() is called instead.

On the dart side, in main.dart the UI is defined, using

ValueListenableBuilder(
        valueListenable: stateHandler.items

to handle UI updates.

The initialization with

late final StateHandler stateHandler;
Future<void> main() async {
  stateHandler = await StateHandler.init();
  runApp(const MyApp());
}

is important as well (although one could do it on the rust side only, using OnceLock).

In my design only state_handler.dart is using the frb generated functions. This is because I use events and effects in my app - but not in this simplified code.

Here the important parts are the initializations:

  static late final StateHandler _singleton;
  static bool _isInitialised = false;

  late final AppState appState;
  late final ValueNotifier<List<Heavy>> items;
  late final ValueNotifier<Heavy?> lastItem;

  static Future<StateHandler> getSingleton() async {
    if (_isInitialised) {
      return _singleton;
    }
    await RustLib.init();
    var appState = await AppState.newInstance();
    var lastItem = await appState.getLastItem();
    _singleton = StateHandler._singletonConstructor(appState, lastItem);
    _isInitialised = true;
    return _singleton;
  }

and the rust function calls

  Future<void> addItem({required int item}) async {
    await appState.addItem(item: item);
    //trigger rebuilds
    items.value = appState.items;
    lastItem.value = await appState.getLastItem();
    print("added item $item, list is now ${itemsToString()}\n");
  }

  Future<void> removeItem({required int index}) async {
    await appState.removeItem(index: index);
    //trigger rebuilds
    items.value = appState.items;
    lastItem.value = await appState.getLastItem();
    print("removed item at index $index, list is now ${itemsToString()}\n");
  }

There are just this questions open:

1. can we get rid of the Clone requirement? Maybe using & with Lifetimes ... but we still want to have Arc<Mutex> under the hood ... -> I assume that the compile doesn't understand that RustAutoOpaque<Heavy> is indeed Arc<Mutex<Heavy>>, dereferences automatically and things Heavy.clone() is called, which isn't (as the panic! is not triggered).

2. What is the difference between RustAutoOpaque and RustOpaque?

3. I couldn't get loging (from Rust) working. IMHO the documentation could be improved. But this is for another discussion ;)

[fzyzcjy]

What I actually would like to have is pointers from Dart to data managed by Rust ... most likely on the heap, but the Rust heap.
In summary, to get to zero-copy read access we need a pointer into shared memory (that is what FRB does with non-opaque types, right?) and a Mutex (what RustAutoOpaque utilizes).

Yes, Dart does support converting a raw pointer into a utf8 string or a uint8list etc. To implement this, we should be quite careful about locks and concurrency. For example, if your Rust code mutates that underlying string/bytes, what should you expect to see in Dart? On the contrary, if you mutate the data in Dart, you will probably violate the Rust memory model (this may be avoided by wrapping and only expose read APIs). Feel free to discuss/PR for this!

[patmuk]

To implement this, we should be quite careful about locks and concurrency. For example, if your Rust code mutates that underlying string/bytes, what should you expect to see in Dart? On the contrary, if you mutate the data in Dart, you will probably violate the Rust memory model (this may be avoided by wrapping and only expose read APIs). Feel free to discuss/PR for this!

Yes ... from the needed behavior it is quite clear - but I don't know how to make this happen with Rust's borrowing system ...

Basically a struct (no matter the actual content) should be manipulated in Rust only, but read by Dart.
And I would like to avoid copy, as that item could be quite heavy (like a video).

So, changes in Rust should update the object in Dart. In my current approach with ValueNotifier/Listener (though that is just the UI triggering part).

I failed with my approaches to get to that so far ... but I am realizing that this might actually be because of Rusts borrow system. What I want is a read-borrow for the Value in Dart, but a write-borrow (mut) in Rust.
And those cannot exist at the same time, as we know.

A minimal Example:

struct MyData {
   myField: Heavy,
}

pub fn mutateHeavy(& mut myField ) -> &myField {
   process(myField)
}

To mutate myData a & mut is needed. To avoid copy & myField is returned.
Now, first mutating the date with a & mut and then returning a & to Dart should be possible (though I did not get there yet) - but I wonder if the & is dropped only when Dart's GC runs?

It doesn't look like Flutters .dispose() is destructing the object, as dart doesn't has destructors.

Thus, I wonder if a & reference in Dart will block Rust from getting a & mut reference for a subsequence call, until Dart's GC runs? (probably not 'block', but panic, when using Arc).

So, by avoiding copy() to be faster we might end up waiting for Dart's GC ...?

@fzyzcjy am I wrong here? As I could not manage yet to get a & to Dart I can't verify.

If I am right my quest for zero-copy is flawed. A valid alternative might be to copy only what Dart would display, I guess with:

struct myData {
   myField: RustAutoOpaque<Heavy>,
}

pub fn mutateHeavy(& myField ) -> &myField {
   process(myField.write().unwrap())
  myField
}

impl Heavy{
  fn get_title(&self) -> String {}
  fn get_interesting_property_to_display(&self) -> usize {}
}

So that Dart only calls the getters. (Alternatively the interesting properties to display in Dart could be put in a newly created struct - the point is that we hand over owned objects to Dart, and not borrowed ones.

In summary, do you think returning & to Dart and aquiring & mut of the same object in Rust for subsequent calls is independent of the GC - or should one avoid that and create/copy objects to give Dart owned data?

[fzyzcjy]

myField: Heavy

This approach will only work for Rust types that can be directly casted to Dart types without changing memory layouts. Thus, Uint8List and String will work, while arbitrary structs may not be able to work. Thus this may be of limited usefulness.

valid alternative might be to copy only what Dart would display

Yes that's pretty reasonable

In summary, do you think returning & to Dart and aquiring & mut of the same object in Rust for subsequent calls is independent of the GC - or should one avoid that and create/copy objects to give Dart owned data?

Just follow the Rust rules. If you &mut on the object, then as long as the returned & is not disposed, your original object cannot be used anymore, just like normal Rust

---

A possible naive implementation:

Let's say we have a Uint8List data. Then I guess the solution is like:

pub struct LockedRustAutoOpaqueUint8List<'a>(&'a Vec<u8>);

fn lockAndReadTheContentWithoutCopy<'a>(arg: &'a Vec<u8>) -> LockedRustAutoOpaqueUint8List<'a> {
  arg
}

using the "lifetime" feature. (I guess some details still need to be tweaked)

And then, make LockedRustAutoOpaqueUint8List implement the Uint8List interface via directly reading the raw pointers. We are safe as long as we only expose a read-only ephemeral interface, since as long as LockedRustAutoOpaqueUint8List is alive (not disposed, we hold a read lock)

[patmuk]

Many thanks for the great answer! I finally understand how this works now :)

myField: Heavy

only work for Rust types that can be directly casted to Dart types without changing memory layouts.

valid alternative might be to copy only what Dart would display

Yes that's pretty reasonable

I overlooked the memory compatibility! So, the primitive types are usually cheap to copy, and often implement copy already. And for really heavy things, like videos, one can use a serialization to Uint8!
I will follow this approach: Copy for trivial stuff, RustAutoOpaque and copy only attributes, which Dart needs to show (e.g. filename of a video), and your described LockedRustAutoOpaqueUint8List approach with ser/de for heavy stuff that got mutated by Rust (e.g. edited videos).

Though the last part I will not do now, but when I really need it (which is probably in some years).

@Vollbrecht I will update my example repo with this approach.

Many thanks again!!!

[fzyzcjy]

You are welcome!

[Vollbrecht]

@patmuk thanks for keeping the topic alive. Today i got also back a bit onto the topic. I inspected the ui_state and ui_mutation attributes and i now think i understand whats the trick there was. Basically they RustState you create will get a inner RustStateBase attached to it. This RustStateBase carry's a async Stream of type () with it.

This Stream will be used to trigger rebuilds of the ui, by registering itself in a top most StatefullWidget. From the dart site of things you then can just use the rust state directly and it will internally using the inner onMutate method by calling add on the Stream of type ().

If i understand correctly if you want to trigger now a rebuild of the ui state that you change from with in your rust code, you would have to manually call the onMutate on the RustStateBase.

That would certainly work for a simple App, though overall it completely sitesteps the intelligent rebuilding of flutter if i understand correctly because it re-triggers a full rebuild on every state change? Maybe that is not a problem its just a observation.

In the coming days i will go deeper here into the topics and also report back on how i will handle it in my case. @patmuk with your approach you still got me a lot to think about, thanks for that.

[fzyzcjy]

though overall it completely sitesteps the intelligent rebuilding of flutter if i understand correctly because it re-triggers a full rebuild on every state change

Yes, that's why this is a super minimal simple demo. This is OK for simple apps (since your UI is small), and for more complex ones you can do more complex solutions. A simple extension may be have more than one state.

[patmuk]

@Vollbrecht ah, great timing :) It is good that both of us are working on this :)

Feel free to ask me anything about my approach once you have questions :)

In a Nutshell, I reduced as much boilerplate as I could, utilizing the ValueNotifier to wrap data, which needs to be triggered for any update, and ValueListeners to trigger UI rebuilds in only the sub-widgets listening to that.

Thanks for looking deeper into @fzyzcjy's approach!

though overall it completely sitesteps the intelligent rebuilding of flutter if i understand correctly because it re-triggers a full rebuild on every state change?

I think that it does not completely sidesteps it: I assume that Flutter still reduces the widget tree to the visible widgets before rendering. But yes, all visible widgets would be rendered, no matter where in the tree.

[fzyzcjy]

I think that it does not completely sidesteps it: I assume that Flutter still reduces the widget tree to the visible widgets before rendering. But yes, all visible widgets would be rendered, no matter where in the tree.

Yes (and a bit more). Flutter has several trees and phases. Triggering such a full setState only causes a rebuild of the widget tree. However, for unchanged parts, it will auto realize it and do not do the heavy re-layout or re-render. For example, suppose we setState but only text of one button is changed, then only that one is re-layout (if needed) and re-rendered, while all the rest skip that heavy phase.