pickle frontend payloads

[benedikt-bartscher]

Benefit from pickles referential serialization to reduce reflex event payload size
This is especially useful when dealing with complex sqlalchemy models which can contain circular and self-referencing relationships
Serialization should also be faster? (did not measure the impact yet)

We could drop most of the serializers and rely on pickle. You can test this with bare sqlalchemy models like this:

from __future__ import annotations

import reflex as rx
from reflex.utils.serializers import serializer
from sqlalchemy import ForeignKey
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship


@rx.model.ModelRegistry.register
class Base(DeclarativeBase):
    pass


@serializer
def serialize_base(obj: Base) -> Base:
    return obj


class TestModel(Base):
    __tablename__: str = "test"

    id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
    name: Mapped[str] = mapped_column(default="test")
    circular_id: Mapped[int] = mapped_column(ForeignKey(f"{__tablename__}.id"))
    circular: Mapped[TestModel] = relationship()


class TestObj:
    name: str
    value: int
    big_json = dict[int, int]

    def __init__(self, name: str = "test", value: int = 0):
        self.name = name
        self.value = value
        self.big_json = {i: i for i in range(1000)}


@serializer
def serialize_testobj(obj: TestObj) -> TestObj:
    return obj


class State(rx.State):
    testmodel: TestModel = TestModel(name="testmodel")

    testobj: TestObj = TestObj()
    testlist: list[TestObj] = []

    def make_circular(self) -> None:
        testmodel = TestModel(name="circular")
        testmodel.circular = testmodel
        self.testmodel = testmodel

    def append(self) -> None:
        self.testlist.append(self.testobj)


def json_var(var: rx.Var) -> rx.Component:
    """Display a variable as a JSON string"""
    return rx.container(
        rx.text(f"Debug {var._var_name}, Type {var._var_type}"),
        rx.text(
            var._replace(
                _var_name=f"JSON.stringify({var._var_full_name}, null, 4)",
                _var_type=str,
                _var_full_name_needs_state_prefix=False,
            )
        ),
    )


def index() -> rx.Component:
    return rx.container(
        rx.container(
            rx.text(
                "Notice how the state payload size does not increase much, because only references are added"
            ),
            rx.button("append", on_click=State.append),
            rx.foreach(State.testlist, lambda obj: json_var(obj.name)),
            rx.text(f"contains: {State.testlist.contains(State.testlist[0])}"),
            rx.text(f"equals: {State.testlist[0] == State.testlist[1]}"),
        ),
        # json_var(State.testobj),
        rx.divider(),
        rx.container(
            rx.button("make circular", on_click=State.make_circular),
            json_var(State.testmodel.name),
            json_var(State.testmodel.circular.name),
            json_var(State.testmodel.circular.circular.name),
        ),
    )


app = rx.App()
app.add_page(index)

Advantages:

• Smaller payloads
• Faster serialization
• Referential Serialization (allow recursion)
• Custom serializers are optional, most objects should just work fine out of the box.
• Object identities in js. Should allow for easier comparison on client side. For example, if an array of objects contains a specific object which currently does not work, because [{a:1}].includes({a:1}) is false in js

Disadvantages:

• Payloads are not human-readable anymore
• dependency on pickleparser, but we could implement our own in the future

[benedikt-bartscher]

This is still draft, because test_upload_file fails.
I will fix it once the reflex team leaves initial feedback for the idea.