Relm Interceptors: a proposal for data store support (and data store-like things)

[chriskilding]

Many apps need to read or write state from an external location in order to be useful. This external location could be, for example, a remote Web API, or a database, or a local file on disk.

This is a proposal for a Relm feature that could support data stores - and other things which behave like them.

At this point I am just putting some ideas out there to get the ball rolling; this is by no means the final state of the feature.

Considerations

We must account for the following requirements in this feature.

Data model != View model

The data model (the structure of the state that is read/written) may differ significantly from the view model (the Relm4 Component hierarchy in your app, or the Self::Init types within Components). Therefore, any data store feature for Relm must be able to translate between the two.

Separation of concerns

Relm4 Components should not have direct knowledge of the data store. Among other things, this means the data store class should not appear as an instance variable or method on components (or be passed within a Component's Self::Init). Essentially it should be possible to attach or detach the interceptor (data store functionality) to your Relm app without your Relm views knowing.

Event-based updates

Traditional data stores (such as local files) can be written to in a monolithic manner (prepare entire data model in-memory and serialize the whole thing when 'Save' is invoked), but this is not efficient for large files. Newer data stores, such as real-time Websockets APIs, cannot be used this way at all; they are event-based. Therefore, any data store feature for Relm should have an event-based update API.

For example, if I have a to-do app with the following data model, if I update the text of a Todo within a List, I should be able to issue a delta update in the backend for just that individual changed item (e.g. an HTTP request PATCH /lists/1/todos/2):

// view model

Lists(
    lists: [
        List(
            name: "Shopping",
            todos: [
                Todo(
                    text: "Bread",
                    is_done: false,
                ),
                Todo(
                    text: "Milk",
                    is_done: true,
                )
            ]
        ),
        List(
            name: "Tasks",
            todos: [
                Todo(
                    text: "foo",
                    is_done: false,
                ),
                Todo(
                    text: "bar",
                    is_done: false,
                )
            ]
        )
    ]
)

// events for the view model

enum ListsInput {
    PushList,
    PopList
}

enum ListInput {
    UpdateName(String),
}

enum TodoInput {
    UpdateText(String),
    UpdateDone(bool),
}

Proposed solution

I propose adding a new "interceptor" feature to Relm4.

The interceptor should be able to intercept any Input event within the view model hierarchy that it's attached to.

For example, in the to-do app view model shown above, if I attach the interceptor at the Lists level, it should be able to intercept ListsInput ListInput and TodoInput events. Meanwhile if I attach it at the List level, it should only intercept ListInput and TodoInput events.

The interceptor should (somehow) delegate to existing Relm features for async work (such as Worker), to handle the actual interaction with the backend.

In other words, Relm4 already has Components (with events) and Workers (to do the data store I/O); the interceptor is there to plug the two together.

Benefits

In addition to satisfying the requirements above, the interceptor feature has the following benefits.

• You can attach more than just a data store. Using the very same mechanism, you could (for example) attach a logger.
• It encapsulates the mode of reading/writing. Your I/O implementation behind the interceptor could maintain a monolithic in-memory data model, modify it when updates are received, then write the whole thing when Save is called. But it could equally issue delta updates to a Web API. Or it could store a change data capture queue (basically a queue of <Foo>Updated type events), batch multiple update events together, then sync the update stream to the backend periodically.

Open questions

Reading plus writing (and preventing cycles)

The proposal so far mainly covers how an interceptor would support writes to a data store (view model -> data model). We also need it to support reads (data model -> view model). If we want the data -> view updates to use the existing Relm Component input events mechanism, the interceptor will need to know the originator of an event. This is to prevent infinite cycles of reading and writing.

In the example above, say that you have an interceptor that reacts to TodoInput events. When the TodoInput::UpdateText event occurs, it should write the new value to the data store. When a change occurs in the data store, the interceptor should issue a TodoInput::UpdateText event back to the view. Left unchecked, this would create a cycle. But if the interceptor knows that it was the originator of the TodoInput::UpdateText event, it can then decide to stop further processing, and not write that event's value to the data store. This breaks the cycle.

Using interceptors in unit/integration tests

Another question is whether we could use the interceptor in unit/integration tests for a Relm app. This would allow us to observe events that are going on within the app. It could also allow us to raise fake events for the test.

Handling failures

Attached interceptors might fail to handle an event.

Sometimes we don't care about the failure (e.g. if a log line doesn't get written, that's probably not critical), so we can just let that happen silently.

But sometimes we do care about the failure (e.g. if the app's remote API is down, and after retrying-with-backoff the app still can't contact it), and we might need to alert the user or prevent further processing. The interceptor feature will need a mechanism to do this.