Debugging Local First Apps with Dexie
What is a local-first app, and how is Dexie related?
In recent years, there has been a growing interest in local-first apps due to data privacy, security, and user control concerns. Local-first apps provide a more secure and responsive user experience by keeping data on the client-side. Local-first apps are also amazing on the UX side of things, all the app data is accessible and manageable when offline and loads faster than non-local first apps. However, this approach also comes with unique challenges, particularly when it comes to debugging the client-side database.
The basic idea of a local-first app is an app that does not rely on the network for its functionalities. It handles its operations with a local-DB that is followed by a syncing process between the frontend and the backend (also between different clients). The syncing concepts, problems, and solutions are a large subject and will be not covered here.
There are many solutions for local-DBs: WatermelonDB, RXDB, SQLite, Automerge, etc. Here, we will discuss the Dexie local-DB. Dexie is a wrapper for indexedDB, used for browser-based clients. Dexie is a minimalist and useful local-DB solution that makes querying, migrating, live data hooks, etc easier and more intuitive.
When developing a local-first app, you maintain a DB on the client, which is one of the most sensitive parts of any app, especially when stored locally. Local-DB solutions can sometimes not be as robust as well-known and used backend DBs systems, and they also generally have smaller product or open-source ecosystems. All of these make the debugging of your Local-DB harder and more important. This is the heart of your local-first app, you should pay attention to it!
Debugging web apps, console.log is your best friend
When debugging web apps your best friend is the browser DevTools, the two primary tools for debugging a web app are logs (info logs, error logs, network logs, etc) and the debugger tool that creates a breakpoint.
However, when working with Dexie, logging operations against the DB can be particularly challenging. Unlike traditional backend-based DB solutions, there is no built-in way to log the operations that take place against the DB. This can make debugging and understanding the app’s behavior a difficult and time-consuming task, especially when trying to catch bugs related to data changes in the Dexie storage.
dexie-logger to the rescue
To solve the issues described above, we created a new npm package called dexie-logger. This package is a simple middleware that plugs into your existing Dexie definitions and offers you intuitive and useful logging powers.
The package uses Dexie’s middleware system and logs the operation name and content provided from the middleware props, along with other stats and info.
Here’s an example of simple logging middleware in Dexie:
db.use({
stack: 'dbcore',
name: 'LoggingMiddleware',
create(downlevelDatabase) {
return {
...downlevelDatabase,
table(tableName) {
const downlevelTable = downlevelDatabase.table(tableName);
return {
...downlevelTable,
mutate: (req) => {
// Important lines HERE
console.log('Mutate operation');
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
return downlevelTable.mutate(req);
},
};
},
};
},
});
Using the logger
To use the logger simply add the middleware to your DB:
// Import the logger
import logger from 'dexie-logger';
// Apply the logger
db.use(logger());
You can also whitelist / blacklist specific tables or operations:
// Import the logger
import logger from 'dexie-logger';
// Apply the logger
db.use(
logger({
tableWhiteList: ['events', 'users'],
operationsWhiteList: ['mutate', 'query'],
}),
);
The logs will look like this:
You can see the operation type, the table name, the operation time, and the specific operation details and response contents.
How we have used it
At 10Play one of the things that we are working on is building full-stack solutions for local-first apps. One of our biggest projects is Skiff calendar — an e2ee (end-to-end encrypted) calendar that is part of a e2ee, privacy-first workspace. These are two simple examples of how we used the logger in the skiff calendar project.
Overwritten Value
In this case, we saved a draft-event object, with an important field — updatedAt , to Dexie. We logged the content that we are writing to Dexie in the draftUpdate function. And logged the content of the saved draft right after. The weird thing that we saw with those logs is that the correct value is getting written to dexie but the wrong value is being read.
So we enabled the dexie-logger, with a specific whitelist for that table and the suspected operations.
db.use(
logger({
tableWhiteList: ['drafts'],
operationsWhiteList: ['mutate', 'get'],
}),
);
Then we did the action again and saw something interesting. We saw two put operations and only after that a get. That means we have some kina of side effect outside the draftUpdate function that is overwriting the updatedAt with bad values.
After we discovered the cause, we simply looked for the places in the code that we write to the draft table and found the ones with the bad content.
Metadata read optimization (React + Apollo client reactive vars)
When enabling the logger on all IndexedDB tables (without any options), we saw a concerning amount of get calls to a table that holds metadata of the current user, which rarely changes) This metadata holds the a user’s UUID and the checkpoint for a recent local-first sync. Both are used in a lot of places in the code. The old code looked something like that:
const getUser = async () => {
// Query to dexie
return db.user.toArray()[0];
};
const someFunction1 = async () => {
const userID = (await getUser()).id;
// Does something
};
const someFunction2 = async () => {
const userID = (await getUser()).id;
// Does something
};
const someFunction3 = async () => {
const userID = (await getUser()).id;
// Does something
};
What happens here is that every time you call each of the functions there is a query to Dexie. That is bad because even if they are low cost the Dexie reads take longer than simply reading some state. And with the number of calls we saw, improving this will have a large impact. We decided to create a centralized state and update it from a single place. This is how we did it:
First, let’s define the userData reactive var, this is a state management solution from Apollo client (you can do the same with plain React Context). In our project, we have already used Apollo client so we decided to stick with it. We are defining a simple saveUserData method, and a hook provided by apollo client API. The most interesting part is the getUserData which we create with a fallback, so if this is called before the value of the user is set we will call dexie instead of returning null.
//userDataVar.ts
const userData = makeVar<User | null>(null);
export const saveUserData = (data: User | null) => {
userData(data);
};
export const getUserData = async () => {
let userData = userData();
// if the var is still not defined try to get it from the db and update the var
if (!userData) {
userData = await getUser();
if (userData) saveUserData(userData);
}
return userData;
};
export const useUserData = () => useReactiveVar(userData);
Now, let’s make the custom hook. This is a simple react custom hook that will keep the userData updated. We are using useLiveQuery from dexie-react-hooks which will rerun the query on changes and keep the value up to date. Then the useEffect calls the saveUserData when the value changes.
// useUserDataUpdate.ts
const useUserDataUpdate = () => {
const userData = useLiveQuery(() => getUser(), []) || null;
useEffect(() => {
saveUserData(userData);
}, [userData]);
};
export default useUserDataUpdate;
Now we simply use the new function
const someFunction1 = async () => {
const userID = (await getUserData()).id;
// Does something
};
const someFunction2 = async () => {
const userID = (await getUserData()).id;
// Does something
};
const someFunction3 = async () => {
const userID = (await getUserData()).id;
// Does something
};
These two cases are simple examples of how the logger can help you detect and solve problems. We have more interesting problems and more advanced dexie-logger use cases coming up.
If you are developing a local-first app with Dexie or just using Dexie for your project. dexie-logger should just be a part of your toolkit. This will make your developing experience better and your app less bug-prone.
What’s next?
We are working on a new version of dexie-logger that will have more tooling also for analyzing and DB optimizations. When it is stable we will release it to the open-sourced repo. This and more optimizations in our next post.
