Profile service

Data storage

React/Redux put an application's state into a single object tree. Actions and reducers move from one state to another, and components respond to the resulting state changes.

This poses three obvious problems when building a rich client application.

Firstly, this state has to fill multiple needs. It's the canonical state of the application, so it needs to be persisted, but it also needs to support the various representations used in the application — search, lists, tables, detail views, etc.

Part of our goal is to go further. It's hard to imagine a single representation that is, for example, easy to free-text search, easy to search geospatially, a close match for typical list UIs, has a reasonable memory footprint, and isn't absurdly expensive to update. We already know from projects like Lovefield that in-memory JS objects can't beat on-disk SQLite indices on rudimentary relational queries, let alone truly rich queries.

Secondly, this state is going to get big. Ten thousand URLs, one hundred thousand visits, locations, favicons, thumbnails, more. Quite soon it becomes impractical to keep the entire state — the entire profile — in memory, periodically flushed to disk.

Thirdly, this state needs to change based on external events: Sync and push, for example. Processes like Sync make the action/reducer/state model really complicated, because they're large-scale, create and resolve conflicts, and need to see a consistent view of the world for protracted periods of time.

Fortunately, Reacty web applications already have a model for this kind of division. The application itself has a state that's oriented towards the needs of the UI, and a backend service owns the full event- or relational-based state for the user. Actions poke the backend service, and the backend service sends canonical state-changing actions down to the app.

Taking Facebook as an example: the client-side JS doesn't keep a local copy of every post, every user, and every like. Clicking the Like button on a story updates the backend, proactively bumps some counters locally, and deals with the fallout later if the backend gives a conflicting update (e.g., the story was deleted).

We can take a similar approach for Tofino. Our app state is everything we need to display UI: open tabs, back stack, a Bloom filter of visited UIs, the results of your current history search. A profile service stores everywhere you've ever been and all of your bookmarks.

Sync works directly against the profile service. The app uses the profile service as its source of truth, with the app state being a UI-centric temporary interpretation plus UI-related persistent state (e.g., window positions).

Initially we'll host the profile service in the Electron main process. UI processes communicate with it over IPC. (We have the option of intermediating via the main process; doing this, and by implication sharing app state between windows, has advantages and disadvantages.)

This has some nice properties:

• The profile service is entirely in control of the mutation of its state. It can go so far as to serialize and pause writes during a sync. We can switch storage mechanisms without affecting clients.
• The profile service can stay resident in memory while the foreground application is closed. This is a model we should adopt for updates, push notifications, et al.
• We can finally sanely support sign-in-to-browser and profile-in-the-cloud: models that require a coherent definition of a profile that's independent from the running application.
• We can, if we're feeling masochistic, switch out IPC for alternative models: true network operation, local HTTP, whatever. Defining a profile storage API would even allow us to use v2's UI with a v1 profile.