Filter data synchronization
In this tutorial you modify networkSetup.ts
to filter the information you synchronize.
Filtering information this way allows you to reduce the use of network resources and makes loading times faster.
MUD initial data hydration, and therefore filtering, comes in two flavors: Dozer and generic. Note that this is for the initial hydration, currently limits on on-going synchronization are limited to the generic method.
Dozer | Generic | |
---|---|---|
Filtering | Can filter on most SQL functions | Can only filter on tables and the first two key fields (limited by eth_getLogs (opens in a new tab) filters) |
Availability | Redstone (opens in a new tab), Garnet (opens in a new tab), or elsewhere if you run your own instance | Any EVM chain |
Security assumptions | The Dozer instance returns accurate information | The endpoint returns accurate information (same assumption as any other blockchain app) |
Why are only the first two key fields available for filtering?
Ethereum log entries can have up to four indexed fields (opens in a new tab). However, Solidity only supports three indexed fields (opens in a new tab) because the first indexed field is used for the event name and type. In MUD, this field (opens in a new tab) specifies whether a new record is created (opens in a new tab), a record is changed (either static fields (opens in a new tab) or dynamic fields (opens in a new tab)), or a record is deleted (opens in a new tab). The second indexed fields is always the table's resource ID. This leaves two fields for key fields.
Setup
To see the effects of filtering we need a table with entries to filter. To get such a table:
Filtering
Edit packages/client/src/mud/setupNetwork.ts
.
- Import
pad
(opens in a new tab) from viem. - Add a
filters
field to thesyncToRecs
call.
/*
* The MUD client code is built on top of viem
* (https://viem.sh/docs/getting-started.html).
* This line imports the functions we need from it.
*/
import {
createPublicClient,
fallback,
webSocket,
http,
createWalletClient,
Hex,
ClientConfig,
getContract,
} from "viem";
import { encodeEntity, syncToRecs } from "@latticexyz/store-sync/recs";
import { getNetworkConfig } from "./getNetworkConfig";
import { world } from "./world";
import IWorldAbi from "contracts/out/IWorld.sol/IWorld.abi.json";
import { createBurnerAccount, transportObserver, ContractWrite } from "@latticexyz/common";
import { transactionQueue, writeObserver } from "@latticexyz/common/actions";
import { pad } from "viem";
import { Subject, share } from "rxjs";
/*
* Import our MUD config, which includes strong types for
* our tables and other config options. We use this to generate
* things like RECS components and get back strong types for them.
*
* See https://mud.dev/templates/typescript/contracts#mudconfigts
* for the source of this information.
*/
import mudConfig from "contracts/mud.config";
export type SetupNetworkResult = Awaited<ReturnType<typeof setupNetwork>>;
export async function setupNetwork() {
const networkConfig = await getNetworkConfig();
/*
* Create a viem public (read only) client
* (https://viem.sh/docs/clients/public.html)
*/
const clientOptions = {
chain: networkConfig.chain,
transport: transportObserver(fallback([webSocket(), http()])),
pollingInterval: 1000,
} as const satisfies ClientConfig;
const publicClient = createPublicClient(clientOptions);
/*
* Create an observable for contract writes that we can
* pass into MUD dev tools for transaction observability.
*/
const write$ = new Subject<ContractWrite>();
/*
* Create a temporary wallet and a viem client for it
* (see https://viem.sh/docs/clients/wallet.html).
*/
const burnerAccount = createBurnerAccount(networkConfig.privateKey as Hex);
const burnerWalletClient = createWalletClient({
...clientOptions,
account: burnerAccount,
})
.extend(transactionQueue())
.extend(writeObserver({ onWrite: (write) => write$.next(write) }));
/*
* Create an object for communicating with the deployed World.
*/
const worldContract = getContract({
address: networkConfig.worldAddress as Hex,
abi: IWorldAbi,
client: { public: publicClient, wallet: burnerWalletClient },
});
/*
* Sync on-chain state into RECS and keeps our client in sync.
* Uses the MUD indexer if available, otherwise falls back
* to the viem publicClient to make RPC calls to fetch MUD
* events from the chain.
*/
const { components, latestBlock$, storedBlockLogs$, waitForTransaction } = await syncToRecs({
world,
config: mudConfig,
address: networkConfig.worldAddress as Hex,
publicClient,
startBlock: BigInt(networkConfig.initialBlockNumber),
filters: [
{
tableId: mudConfig.tables.app__Counter.tableId,
},
{
tableId: mudConfig.tables.app__History.tableId,
key0: pad("0x01"),
},
{
tableId: mudConfig.tables.app__History.tableId,
key0: pad("0x05"),
},
],
});
return {
world,
components,
playerEntity: encodeEntity({ address: "address" }, { address: burnerWalletClient.account.address }),
publicClient,
walletClient: burnerWalletClient,
latestBlock$,
storedBlockLogs$,
waitForTransaction,
worldContract,
write$: write$.asObservable().pipe(share()),
};
}
Click Increment a few times to see you only see the history for counter values 1 and 5. You can also go to the MUD Dev Tools and see that when you select Components > app__History it only has those lines.
Explanation
The filters
field contains a list of filters.
Only rows that match at least one line are synchronized.
Each filter is a structure that can have up to three fields, and all the fields that are specified must match a row for the filter to match.
tableId
, the table ID to synchronize. You can read this value frommudConfig.tables
.key0
, the first key value (as a 32 byte hexadecimal string).key1
, the second key value (as a 32 byte hexadecimal string).
The filters in the code sample
filters: [
{
tableId: mudConfig.tables.app__Counter.tableId,
},
The first filter is for the app__Counter
table (Counter
in the app
namespace).
We don't specify any keys, because we want all the rows of the table.
It's a singleton so there is only one row anyway.
{
tableId: mudConfig.tables.app__History.tableId,
key0: pad("0x01"),
},
{
tableId: mudConfig.tables.app__History.tableId,
key0: pad("0x05"),
},
],
These two filters apply to the History
table.
This table has just one key, the counter value which the row documents.
We need a separate filter for every value, and here we have two we care about: 1
and 5
.
Limitations
There are several limitations on filters.
- We can only filter on these fields:
- The table ID (
tableId
) - The first key (
key0
) - The second key (
key1
)
- The table ID (
- We can only filter by checking for equality. We cannot check ranges, or get all values except for a specific one (inequality).
Of course, once we have the data we can filter it any way we want. The purpose of these filters is to restrict the information we get at all, either directly from the blockchain or from the indexer.