React Loki and XState
If you'd just like to see the code, it's here: https://github.com/DaveWelling/rlx
At my work, we've built a progressive web app for data entry. Nothing exciting there except that the application uses metadata to dynamically generate experiences for a broad range of use cases. You may be asking yourself why in the world somebody would do that.
In our case, we have a lot of different clients with slightly different requirements around data capture. At first, we would assign teams to each new client with instructions to build a custom application but reuse code from previous implementations, upgrading that code as necessary. As you might be imagining, this was a nasty maintenance disaster.
For years, there's been a lot of talk, both positive and negative, about 'low-code' development platforms, and I don't think that is what we've built. Our effort is more like one application which we've made incredibly configurable. It transforms and gives different experiences depending on who logs in, or more specifically, what application "tenants" and use cases the user can access.
I'm telling you all this to provide some context for what I'd like to demonstrate. Often, a very large react app will use something like redux to manage state... and ours does. Unfortunately, we've run into a problem that many users of Redux have hit - we've used it for too much. As I mentioned, our application uses metadata extensively to determine how it is rendered, and how the corresponding data is manipulated. This creates an interesting dynamic where metadata informs the messages that redux sends.
As an example, if our application is submitting a 'foo' to be saved and the 'foo' is grouped in a namespace of data called 'bar'. This is what a simplified version of the redux dispatch might look like:
{
type: 'submit_bar_foo',
submit: {
newModel: {
_id: 'someObjectId',
title: 'some foo title'
}
}
}
and the code to generate that message might look something like this:
{
type: `${verb}_${namespace}_${relation}`,
[verb]: {
newModel: {
_id,
title
}
}
}
How do you even do that? In our case, we've added custom redux middleware and reducers that are dynamically generated based on the content of our metadata. If this sounds like it is not a trivial task and like the code might be complex to maintain, then I've described the situation accurately.
This by itself would be tolerable considering the payoff, but on top of that, we've created middleware to handle many other concerns such as security, persistence (aka writing to a local lokiJs database) and use case customizations (those that did not make sense to implement using metadata).
The result is an amazingly flexible piece of software. We can effectively create an entire new use case and deploy it for testing in a matter of hours. Unfortunately, it takes 6 months (or more) before a new developer on the team is remotely comfortable in the code.
So... I'd like to design something simpler -- or at least easier to understand.
A while back, I realized that one of the major attractions of redux, i.e. centralized state management, is actually getting in the way for us. This is also why using a similar tool like recoil.js or mobx or whatever, probably would not have worked for us. We've already got a centralized state management mechanism: we're using lokiJs to store our data in indexedDB. By adding redux or a similar tool to moderate state between the lokiJs database and the code, it starts to feel like we've strayed into ORM-land. Part of why I moved away from C# and SQL Server was to escape ORMs. It's not that ORMs are bad (in all situations), but there is no way you need that level of complexity when you have have an application that uses data in a way that is almost identical to that in which it is stored. I'm digressing. The point is: the combination of LokiJs with react hooks is more than sufficient to moderate state transformations in our client side app.
The problem is that lokiJs doesn't come with a built in method of informing react components that the data they are rendering has changed... or does it?
Actually, lokiJs has something called a dynamicView which encapsulates a subset/transformation of your data collections (similar to an SQL view), but it also allows you to observe changes to the dynamicView!
This means that instead of using useSelector
from redux-react. I can create a similar hook called useLokiView:
/**
* Similar to useSelector from react-redux but with a loki backing store
* see https://github.com/reduxjs/react-redux/blob/96bf941751a8460c5cf64027348f05d332e19a20/src/hooks/useSelector.js
* @param collectionName The loki collection
* @param viewName The name of the dynamic view in loki
* @param viewCriteria MongoDB find syntax
*/
export default function useLokiView(collectionName, viewName, viewCriteria) {
// Trick to force a new render when loki reports a change to the view
const [, forceRender] = useReducer((s) => s + 1, 0);
const collection = db.getCollection(collectionName);
let view = collection.getDynamicView(viewName);
// Lazy creation of dynamic view.
if (!view) {
view = collection.addDynamicView(viewName);
view.applyFind(viewCriteria);
}
useLayoutEffect(() => {
// Throttle renders - may need to revisit for something like an
// import of a file which could create many dynamic view rebuilds
const onRebuild = throttle(() => forceRender(), 250);
view.addListener("rebuild", onRebuild);
return () => view.removeListener("rebuild", onRebuild);
}, []);
return [view.data()];
}
If I use this, I don't have to worry about how sorting or filtering the state in one part of my page might affect a different view of that state in another part of the page. LokiJs will do the record keeping for me. Notice the viewName
parameter for the hook? This tells LokiJs that the data being requested is a unique view of the recordset (aka collection). So to drive the point home: if I have two grids displaying the same data, but with different filters, this code will tell LokiJs to keep them separate and to independently report changes for each.
Form state #
One of the dirtiest, meanest parts of our code base is the Form workflow. As you may have guessed, we're using Redux to manage form state. This means that all the UI form component change messages are going through middleware and the code for this sits next to the regular redux state management code.
In retrospect, this is a very bad design. Forms are nasty animals and many open source form libraries have failed. Trying to write your own might be a tough proposition. Here's a video of David Khourshid (the author of XState) which explains why this is difficult and also explains why I decided to try a general purpose state machine library instead.
To summarize one of the main points of the video, I'll say that, in the past, we've done EXACTLY what David describes with adopting an open source form library, finding initial success, but then eventually hating the complexity of introducing customizations. To be fair, because we render data dynamically, ours may actually have been a worse experience than most form library consumers.
This is getting a little long, so I'll create a separate blog post to describe the code in more detail. Before I go, I'd like to point out that this blog and prototype gloss over an important area of the app. At my work we call it "business logic". For us, I suppose it includes most requirements-driven decision making or data storage/forwarding operations.
Removing Redux might have a profound impact on the answers to questions like:
- What happens after you click the 'submit' button?
- Yes, the XState form machine will include a validation step and a step to persist the data, but these two things by themselves could probably take up several blogs (or books if you are trying to synchronize the data with a backend).
- How do you tell the application it is ready to render persisted data or that the user is authenticated?
- Hopefully it is obvious that the approaches in this prototype are naively simple (or absent) and more sophisticated solutions are often mediated by Redux or similar libraries. An alternative approach might be needed if you entirely remove them.
It might be unfair to bring these problems up and not offer any suggestions at all. However because our application is so dynamic, most of these business logic problems are going to be handled by a domain specific, home-grown rules engine which (for us) behaves like an actor model, and I don't think that solution makes any sense at all for most applications. I suspect most applications with straight-forward imperative programming of these problems will be easier to maintain. In any case, that discussion is well beyond the scope of this demo.
- Next: React Loki and XState - Part 2
- Previous: A new Blog.