HyperIndex Complete Documentation
This document contains all HyperIndex documentation consolidated into a single file for LLM consumption.
| What it is | A blazing-fast, developer-friendly multichain blockchain indexer that transforms on-chain events into structured, queryable databases with GraphQL APIs |
| Data engine | Powered by HyperSync - up to 2000x faster than traditional RPC endpoints |
| Performance | Ranked #1 fastest indexer in independent Sentio benchmarks (April 2025) - up to 6x faster than the nearest competitor, 63x faster than TheGraph |
| Supported chains | 70+ EVM chains and Fuel, with new networks added regularly; all EVM-compatible chains supported via RPC |
| Languages | TypeScript, JavaScript, ReScript |
| Key files | config.yaml (indexer settings), schema.graphql (data schema), src/EventHandlers.* (event logic) |
| Prerequisites | Node.js v22+, pnpm v8+, Docker Desktop (local dev only) |
| Deployment | Hosted service (managed, no API token needed) or self-hosted |
| API token | Required for local dev and self-hosted deployments from 3 November 2025 via ENVIO_API_TOKEN env variable |
| Query interface | GraphQL API auto-generated from your schema |
| Multichain | Native multichain indexing with unordered_multichain_mode support |
| Wildcard indexing | Index by event signature rather than contract address |
| Migration | Straightforward migration path from TheGraph subgraphs |
| Get started | pnpx envio init |
| Support | Discord ยท GitHub |
Overviewโ
File: overview.md
HyperIndex is a blazing-fast, developer-friendly multichain indexer, optimized for both local development and reliable hosted deployment. It empowers developers to effortlessly build robust backends for blockchain applications.
HyperIndex is Envio's full-featured blockchain indexing framework that transforms on-chain events into structured, queryable databases with GraphQL APIs.
HyperSync is the high-performance data engine that powers HyperIndex. It provides the raw blockchain data access layer, delivering up to 2000x faster performance than traditional RPC endpoints.
While HyperIndex gives you a complete indexing solution with schema management and event handling, HyperSync can be used directly for custom data pipelines and specialized applications.
Feature Roadmapโ
Upcoming features on our development roadmap:
- Isolated Multichain Mode
- Polished Solana Support
- Indexing 1,000,000+ events per second
๐ Quick Linksโ
Contract Importโ
File: contract-import.md
The Quickstart enables you to instantly autogenerate a powerful blockchain indexer and start querying blockchain data in minutes. This is the fastest and easiest way to begin using HyperIndex.
Example: Autogenerate an indexer for the Eigenlayer contract and index its entire history in less than 5 minutes by simply running pnpx envio init and providing the contract address from Etherscan.
Getting Startedโ
Run the following command to initialize your blockchain indexer:
pnpx envio init
You'll then follow interactive prompts to customize your indexer.
Indexer Initialization Optionsโ
During initialization, you'll be presented with two options:
- Contract Import (recommended for existing smart contracts)
- Template
Choose the Contract Import option to auto-generate indexers directly from smart contracts.
? Choose an initialization option
Template
> Contract Import
[โโ to move, enter to select]
2. Local ABI Importโ
Choose this method if the contract ABI is unavailable from a block explorer or you're using an unverified contract.
Steps:โ
a. Select Local ABI
? Would you like to import from a block explorer or a local abi?
Block Explorer
> Local ABI
[โโ to move, enter to select]
b. Specify ABI JSON file
Provide the path to your local ABI file (JSON format):
? What is the path to your json abi file?
c. Select events to index
? Which events would you like to index?
> [x] ClaimRewards(address indexed from, address indexed reward, uint256 amount)
[x] Deposit(address indexed from, uint256 indexed tokenId, uint256 amount)
[space to select, โ to select all, โ to deselect all]
d. Choose blockchain
Specify the blockchain your contract is deployed on:
? Choose network:
> ethereum-mainnet
goerli
optimism
base
bsc
gnosis
[Custom Network ID]
[โโ to move, enter to select]
e. Enter contract details
- Contract name
? What is the name of this contract?
- Contract address
? What is the address of the contract?
[Use proxy address if ABI is for a proxy implementation]
f. Finish or add more contracts
Complete the import process or continue adding contracts:
? Would you like to add another contract?
> I'm finished
Add a new address for same contract on same network
Add a new network for same contract
Add a new contract (with a different ABI)
Congratulations! Your HyperIndex indexer is now ready to run and query data!
Next step: Running your Indexer locally or Deploying to Envio Cloud.
Quickstart With Aiโ
File: quickstart-with-ai.md
Build an Envio HyperIndex indexer end-to-end with an AI coding assistant.
Most developers now reach for an AI coding assistant before they open a file. This guide walks through an AI-centric flow for creating, developing, and deploying a HyperIndex indexer. It is semi-generic, so any capable AI coding assistant (Cursor, Windsurf, Copilot Agent, Continue, etc.) will work. That said, we've seen the best results with Claude Code and recommend starting there.
If you'd rather drive the CLI yourself, see the Quickstart.
Step 1. Initialize The Indexerโ
Open Claude/Cursor/Codex and prompt:
pnpx envio init
Built for AI Agentsโ
When we notice a command is run by an agent instead of interactively, we output an AI-friendly prompt with the available options and step-by-step instructions on what to do next.
We also provide tools and recommendations an agent can use to get the result, like envio tools search-docs, with more coming soon.
After the project is initialized, we provide a curated set of skills that guide an agent through the codebase. Together with our testing framework, they let it iterate quickly on indexer changes while keeping quality high.
Upgrading Envio or have stale skills? Run envio skills update to pull the latest skills into your project.
About Envio API Tokenโ
The Envio API token is your HyperSync API token. A few things to know:
- The token can't currently be created programmatically. You generate one by logging in to envio.dev/app/api-tokens and copying it into
ENVIO_API_TOKENin your indexer's.env. - It's only required for local development and self-hosted deployments. Indexers running on Envio Cloud get special access and don't need a custom token.
- It's required when using Envio as the data provider (HyperSync). If you only use an external RPC as the data source, no token is needed โ you can pass an empty string to skip the prompt.
- To run
pnpm devlocally, generate a token from the link above and setENVIO_API_TOKENin.envbefore starting the indexer.
See API Tokens and Environment Variables for full details.
Step 3. Migrating an Existing Indexerโ
If you're porting from The Graph, Ponder, or another indexing framework, start with the AI migration workflow. It scales much better than hand-editing handlers.
- Migrate Using AI: the recommended assistant-driven flow. It's written around subgraphs, but the same monorepo-plus-phased-prompt pattern works for Ponder and other frameworks. Point the assistant at the source project plus a freshly scaffolded HyperIndex indexer and let the skills guide it.
- Migrate from The Graph (manual)
- Migrate from Ponder
- Migrate from Alchemy
Related Resourcesโ
- MCP Server
- LLM-friendly docs bundle
- Envio CLI reference
- Envio Cloud CLI
- Migrate Using AI
- HyperIndex v3 migration
What's New in HyperIndex V3โ
File: whats-new-in-v3.md
15 full months have passed since the official HyperIndex v2.0.0. Since then, we have shipped 32 minor releases and multiple patches with zero breaking changes to the documented API. We also received PRs from 6 external contributors, grew from 1 GitHub star to over 470, and saw many big projects rely on HyperIndex.
HyperIndex V3 focuses on modernizing the codebase and laying the foundation for many more months of development. This page describes everything that's new. To upgrade an existing project from V2, follow the Migrate to V3 guide.
New Featuresโ
Unified Handlers APIโ
In V3 all handler registrations now happen through a single indexer value. Contract-specific exports (ERC20.Transfer.handler, UniV3.PoolFactory.contractRegister, etc.) have been removed in favor of indexer.onEvent, indexer.contractRegister, and indexer.onBlock.
Event handlers with indexer.onEvent:
indexer.onEvent(
{
contract: "ERC20",
event: "Transfer",
wildcard: true,
where: ({ chain }) => ({
params: [
{ from: chain.Safe.addresses },
{ to: chain.Safe.addresses },
],
}),
},
async ({ event, context }) => {
// Handler logic
},
);
Dynamic contracts with indexer.contractRegister:
indexer.contractRegister(
{
contract: "UniV3",
event: "PoolFactory",
},
async ({ event, context }) => {
context.chain.Pool.add(event.params.poolAddress);
},
);
Block handlers with indexer.onBlock consolidate across chains in a single call:
indexer.onBlock(
{ name: "EveryBlock" },
async ({ block, context }) => {
// Handler logic
},
);
For chain-specific or interval-based block handlers, use the where callback:
indexer.onBlock(
{
name: "Ranges",
where: ({ chain }) => {
if (chain.id !== 1) return false;
return {
block: {
number: {
_gte: 20_000_000,
_lte: 22_000_000,
_every: 100,
},
},
};
},
},
async ({ block, context }) => {
// Handler logic
},
);
Per-Event Start Blockโ
Handlers can specify custom start blocks per chain via where.block.number._gte, overriding contract and chain configuration:
indexer.onEvent(
{
contract: "UniV4",
event: "Pool",
where: ({ chain }) => {
let startBlock: number;
switch (chain.id) {
case 1:
startBlock = 18_000_000;
break;
case 8453:
startBlock = 2_000_000;
break;
default: {
const _exhaustive: never = chain.id;
return false;
}
}
return {
block: { number: { _gte: startBlock } },
};
},
},
async ({ event, context }) => {
// Handler logic
},
);
CommonJS โ ESMโ
We migrated HyperIndex from CommonJS-only to ESM-only. This enables:
- Using the latest versions of libraries that have long since abandoned CommonJS support
- Top-level await in handler files
Top-Level Awaitโ
Thanks to the migration to ESM, you can now use await directly in handler and other files:
const ZERO_ADDRESS = "0x0000000000000000000000000000000000000000";
// Load data before registering handlers
const addressesFromServer = await loadWhitelistedAddresses();
indexer.onEvent(
{
contract: "ERC20",
event: "Transfer",
wildcard: true,
where: {
params: [
{ from: ZERO_ADDRESS, to: addressesFromServer },
{ from: addressesFromServer, to: ZERO_ADDRESS },
],
},
},
async ({ event, context }) => {
// ... your handler logic
},
);
3x Historical Backfill Performanceโ
Achieved by adding chunking logic to request events across multiple ranges at once. This also fixed overfetching for contracts with a much later start_block in the config, as well as speeding up dynamic contract registration. If you had data fetching as a bottleneck, 25k events per second is now a standard.
Automatic Handler Registration (src/handlers)โ
We introduced automatic registration of handler files located in src/handlers.
Previously, you needed to specify an explicit path to a handler file for every contract in config.yaml. Now you can remove all of the paths from config.yaml and simply move the files to src/handlers. You can name the files however you want, but we suggest using contract names and having a file per contract.
If you don't like src/handlers, use the handlers option in config.yaml to customize it.
The explicit handler field in config.yaml still works, so you don't need to change anything immediately.
RPC for Realtime Indexingโ
Built by an external contributor @cairoeth to allow specifying realtime mode for an RPC data source to embrace low-latency head tracking:
rpc:
- url: https://eth-mainnet.your-rpc-provider.com
for: realtime
In this case, the RPC won't be used for historical sync but will be used as the primary source once the indexer enters realtime mode.
Chain State on Contextโ
The Handler Context object provides chain state via the chain property:
indexer.onEvent(
{ contract: "ERC20", event: "Approval" },
async ({ context }) => {
console.log(context.chain.id); // 1 - The chain id of the event
console.log(context.chain.isRealtime); // true - Whether the indexer entered realtime mode
},
);
Indexer State & Configโ
As a replacement for the deprecated and removed getGeneratedByChainId, we introduce the indexer value. It provides nicely typed chains and contract data from your config, as well as the current indexing state, such as isRealtime and addresses. Use indexer either at the top level of the file or directly from handlers. It returns the latest indexer state.
With this change, we also introduce new official types: Indexer, EvmChainId, FuelChainId, and SvmChainId.
indexer.name; // "uniswap-v4-indexer"
indexer.description; // "Uniswap v4 indexer"
indexer.chainIds; // [1, 42161, 10, 8453, 137, 56]
indexer.chains[1].id; // 1
indexer.chains[1].startBlock; // 0
indexer.chains[1].endBlock; // undefined
indexer.chains[1].isRealtime; // false
indexer.chains[1].PoolManager.name; // "PoolManager"
indexer.chains[1].PoolManager.abi; // unknown[]
indexer.chains[1].PoolManager.addresses; // ["0x000000000004444c5dc75cB358380D2e3dE08A90"]
On indexer restart, reading indexer at the top level of a handler file returns values restored from the database โ including dynamically registered contract addresses โ rather than only what's declared in config.yaml:
// Includes initial + dynamically registered addresses persisted in the DB
console.log(indexer.chains.eth.Pool.addresses);
Conditional Event Handlersโ
Now it's possible to return a boolean value from the where function to disable or enable the handler conditionally.
indexer.onEvent(
{
contract: "ERC20",
event: "Transfer",
wildcard: true,
where: ({ chain }) => {
// Skip all ERC20 on Polygon
if (chain.id === 137) {
return false;
}
// Track all ERC20 on Ethereum Mainnet
if (chain.id === 1) {
return true;
}
// Track only whitelisted addresses on other chains
return {
params: [
{ from: ZERO_ADDRESS, to: WHITELISTED_ADDRESSES[chain.id] },
{ from: WHITELISTED_ADDRESSES[chain.id], to: ZERO_ADDRESS },
],
};
},
},
async ({ event, context }) => {
// ... your handler logic
},
);
Automatic Contract Configurationโ
Started automatically configuring all globally defined contracts. This fixes an issue where addContract crashed because the contract was defined globally but not linked for a specific chain. Now it's done automatically:
contracts:
- name: UniswapV3Factory
events: # ...
- name: UniswapV3Pool
events: # ...
chains:
- id: 1
start_block: 0
contracts:
- name: UniswapV3Factory
address: "0x1F98431c8aD98523631AE4a59f267346ea31F984"
# UniswapV3Pool no longer needed here - auto-configured from global contracts
- id: 10
start_block: 0
contracts:
- name: UniswapV3Factory
address: "0x1F98431c8aD98523631AE4a59f267346ea31F984"
# UniswapV3Pool no longer needed here - auto-configured from global contracts
ClickHouse Storage (Experimental)โ
HyperIndex can now run with multiple storage backends at the same time. Postgres remains the primary database, and entities can additionally be written to a ClickHouse database that is restart- and reorg-resistant. Prometheus metrics carry a storage-name label so you can distinguish backends.
Enable backends in config.yaml and route each entity explicitly via the @storage directive in schema.graphql:
storage:
postgres: true
clickhouse: true
# Stored in both Postgres and ClickHouse
type Transfer @storage(postgres: true, clickhouse: true) {
id: ID!
from: String!
to: String!
value: BigInt!
}
# Stored only in ClickHouse
type Snapshot @storage(clickhouse: true) {
id: ID!
blockNumber: BigInt!
}
Per-entity routing is more verbose but lets you write some entities to Postgres and others to ClickHouse only.
envio dev automatically spins up a ClickHouse Docker container for local development with playground-friendly defaults so you can connect to it without configuring a password. For envio start, provide your own connection via the environment variables ENVIO_CLICKHOUSE_HOST, ENVIO_CLICKHOUSE_DATABASE, ENVIO_CLICKHOUSE_USERNAME, and ENVIO_CLICKHOUSE_PASSWORD.
Envio Cloud currently supports ClickHouse on the Dedicated Plan.
For high-availability ClickHouse setups, HyperIndex supports two additional environment variables:
ENVIO_CLICKHOUSE_REPLICATEDโ set totrueto use replicated table engines.ENVIO_CLICKHOUSE_DATABASE_ENGINEโ override the database engine (for example,Replicated).
Do not run multiple indexers writing to the same ClickHouse database at the same time.
HyperSync Source Improvementsโ
Multiple updates on the HyperSync side to achieve smaller latency and less traffic:
- Server-Sent Events instead of polling to get updates about new blocks
- CapnProto instead of JSON for query serialization
- Cache for queries with repetitive filters - huge egress saving when indexing thousands of addresses
- Improved connection establishment behind a proxy
- Configurable log level support via
ENVIO_HYPERSYNC_LOG_LEVELenvironment variable - Automatic rate-limiting handling on the client side
- Better reconnection logic, logging, and fallbacks for HyperSync SSE and RPC WebSocket height streaming for more stable indexing at the chain head
Fuel Block Handler Supportโ
Block handlers are now supported for Fuel indexing.
Solana Support (Experimental)โ
HyperIndex now supports Solana with RPC as a source. This feature is experimental and may undergo minor breaking changes. Solana exposes its block-stream handler as indexer.onSlot (rather than onBlock) to match Solana's slot-based model.
To initialize a Solana project:
pnpx envio init svm
See the Solana documentation for more details.
pnpx envio init Improvementsโ
- Removed language selection to prefer TypeScript by default
- Cleaned up templates to follow the latest good practices
- Added new templates to highlight HyperIndex features:
Feature: Factory Contract,Feature: External Calls - Pre-configured GitHub Actions workflow for running tests and initialized git repository
- Generated projects include Cursor/Claude skills to support agent-driven development
Block Handler Only Indexersโ
Now it's possible to create indexers with only block handlers. Previously, it was required to have at least one event handler for it to work. The contracts field became optional in config.yaml.
Flexible Entity Fieldsโ
We no longer have restrictions on entity field names, such as type and others. Shape your entities any way you want. There are also improvements in generating database columns in the same order as they are defined in the schema.graphql.
Unordered Multichain Mode Onlyโ
Unordered multichain mode is now the only mode in V3 โ events from different chains are processed in parallel without strict cross-chain ordering, which provides better performance for most use cases. The V2 unordered_multichain_mode option and the multichain: ordered opt-in have been removed.
Preload Optimization by Defaultโ
Preload optimization is now enabled by default, replacing the previous loaders and preload_handlers options. This improves historical sync performance automatically.
TUI Improvementsโ
We gave our TUI some love, making it look more beautiful and compact. It also consumes fewer resources, shares a link to the Hasura playground, and dynamically adjusts to the terminal width.
The TUI now shows an events-per-second indicator during backfill so you can see indexing throughput at a glance.
The TUI is also auto-disabled in CI environments and when running under AI agents, so logs stay clean without manual configuration. The legacy TUI_OFF=true environment variable was renamed to ENVIO_TUI=false.
!TUI
New Testing Frameworkโ
HyperIndex ships a purpose-built testing framework powered by createTestIndexer(). Write tests against the same indexer that runs in production โ no database, no Docker, no manual mock wiring.
The framework integrates with Vitest, replacing the previous mocha/chai setup with a single package that doesn't require configuration by default and includes snapshot testing out-of-the-box. It also provides typed test assertions and utilities to read/write entities in-between processing runs.
Three ways to feed eventsโ
1. Auto-exit โ processes the first block with matching events, then exits. Each subsequent call continues where the last one stopped. Zero config needed.
describe("ERC20 indexer", () => {
it("processes the first block with events", async (t) => {
const indexer = createTestIndexer();
const result = await indexer.process({ chains: { 1: {} } });
// Auto-filled by Vitest on first run โ just review and commit
t.expect(result).toMatchInlineSnapshot(`
{
"changes": [
{
"Transfer": {
"sets": [
{
"blockNumber": 10861674,
"from": "0x0000000000000000000000000000000000000000",
"id": "1-10861674-23",
"to": "0x41653c7d61609D856f29355E404F310Ec4142Cfb",
"transactionHash": "0x4b37d2f343608457ca...",
"value": 1000000000000000000000000000n,
},
],
},
"block": 10861674,
"chainId": 1,
"eventsProcessed": 1,
},
],
}
`);
});
});
2. Explicit block range โ pin to specific blocks for deterministic CI snapshots.
const result = await indexer.process({
chains: {
1: {
startBlock: 10_861_674,
endBlock: 10_861_674,
},
},
});
3. Simulate โ feed typed synthetic events for pure unit tests. No network, no block ranges.
await indexer.process({
chains: {
137: {
simulate: [
{
contract: "Greeter",
event: "NewGreeting",
params: { greeting: "Hello", user: "0x123..." },
},
],
},
},
});
Key capabilitiesโ
- Snapshot-driven assertions โ
result.changescaptures every entity set/delete per block. Pair withtoMatchInlineSnapshotfor auto-generated, reviewable snapshots. - Direct entity access โ
indexer.Entity.get(),.getOrThrow(),.getAll(), and.set()for reading and presetting state. - Real pipeline, real confidence โ tests exercise the full indexer pipeline including dynamic contract registration, multi-chain support, and handler context.
- Parallel test execution via worker thread isolation.
The test indexer also exposes chain information:
const indexer = createTestIndexer();
indexer.chainIds; // [1, 42161]
indexer.chains[1].id; // 1
indexer.chains[1].startBlock; // 0
indexer.chains[1].ERC20.addresses; // ["0x..."]
// Read/write entities between processing runs
await indexer.Account.set({ id: "0x123...", balance: 100n });
const account = await indexer.Account.get("0x123...");
See the Testing documentation for more details.
Podman Supportโ
Beyond Docker, HyperIndex now supports Podman for local development environments. This provides an alternative container runtime for developers who prefer Podman or have it available in their environment.
Nested Tuples for Contract Importโ
The envio init command now supports contracts with nested tuples in event signatures, which was previously a limitation when importing contracts.
PostgreSQL Update for Local Docker Composeโ
The local development Docker Compose setup now uses PostgreSQL 18.1 (upgraded from 17.5).
contractName and eventName on Eventโ
Events now include contractName and eventName fields, making it easier to identify which contract and event you're working with in handlers:
indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event }) => {
console.log(event.contractName); // "ERC20"
console.log(event.eventName); // "Transfer"
},
);
New Official Exported Typesโ
Generated code now exports official generic types for entities, enums, and events. These replace the previous contract-specific type exports:
import type {
MyEntity, // Still exported but Entity is preferred
Entity, // Generic entity type โ use as Entity
Enum, // Generic enum type โ use as Enum (replaces direct MyEnum export)
EvmEvent, // Generic event type โ use as EvmEvent
// Access specific fields: EvmEvent["block"]
} from "envio";
Support for DESC Indicesโ
A nice way to improve your query performance as well:
type PoolDayData
@index(fields: ["poolId", ["date", "DESC"]]) {
id: ID!
poolId: String!
date: Timestamp!
}
RPC Source Improvementsโ
Added polling_interval option for RPC source configuration. Also added missing support for receipt-only fields (gasUsed, cumulativeGasUsed, effectiveGasPrice) that are not available via eth_getTransactionByHash. HyperIndex will additionally perform the eth_getTransactionReceipt request when one of the fields is added in field_selection.
WebSocket Support (Experimental)โ
Experimental WebSocket support for RPC source to improve head latency. Please create a GitHub issue if you come across any problems.
chains:
- id: 1
rpc:
url: ${ENVIO_RPC_ENDPOINT}
ws: ${ENVIO_WS_ENDPOINT}
for: realtime
Prometheus Metrics for Data Providersโ
Added a Prometheus metric to track requests to data providers, providing better observability into your indexer's data fetching patterns.
GraphQL-Style getWhere APIโ
The getWhere query API has been redesigned using GraphQL-style syntax:
// Before
const transfers = await context.Transfer.getWhere.from.eq("0x123...");
// After
const transfers = await context.Transfer.getWhere({ from: { _eq: "0x123..." } });
Additionally, three new filter operators are available following Hasura-style conventions:
context.Entity.getWhere({ amount: { _gte: 100n } })
context.Entity.getWhere({ amount: { _lte: 500n } })
context.Entity.getWhere({ status: { _in: ["active", "pending"] } })
Direct RPC Clientโ
Replaced Ethers.js with a direct RPC client implementation, reducing dependencies and improving performance.
Block Lag Configurationโ
A per-chain block_lag option to index behind the chain head by a specified number of blocks. Replaces the global ENVIO_INDEXING_BLOCK_LAG environment variable. Defaults to 0. This is for advanced use cases โ only use it if you know what you're doing.
chains:
- id: 1
block_lag: 5
Official /metrics Endpointโ
Prometheus metrics are now official. We cleaned up metric names, switched time units to seconds instead of milliseconds, and followed Prometheus naming conventions more closely. Metrics also cover data points previously available only via the --bench feature. A separate /metrics/runtime endpoint with a dedicated Prometheus registry is available for runtime metrics, isolated from the default /metrics endpoint.
Starting from the v3.0.0 release, Prometheus metrics will follow semver and be documented.
Breaking changes:
- Cleaned up metric names and switched time units from milliseconds to seconds
- Removed
--benchsupport โ use the/metricsendpoint instead
Use the new envio metrics CLI command to fetch the Prometheus metrics of a locally running indexer without curling the endpoint manually.
Continue on Config Changeโ
HyperIndex can now keep indexing through some config.yaml changes โ rpc configuration is the first to land โ instead of erroring out on every restart. Where a change is incompatible, the CLI prints exactly which fields were touched and offers two clear options (revert, or envio dev -r to wipe and re-index). More flexibility will be unlocked over time; open a GitHub issue if you need a specific field supported.
Double Handler Registrationโ
It's now possible to register multiple handlers for the same event with similar filters:
indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event, context }) => {
// Your logic here
},
);
indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event, context }) => {
// And here
},
);
Improved Multiple Data-Sources Supportโ
After switching to a fallback source, HyperIndex now attempts to recover to the primary source 60 seconds later. Previously, it would stay on the fallback until the fallback was down or the indexer was restarted. The source selection logic has also been improved for better indexing resilience and stricter enforcement of the realtime mode configuration.
Updated Dev Docker Flowโ
envio dev no longer uses a generated Docker Compose file and manages containers, network, and volumes directly for greater flexibility. For example, disabling Hasura with ENVIO_HASURA now prevents envio dev from pulling the Hasura image. Use envio dev --restart (or -r) to forcefully clear the database even if there are no config changes detected.
Envio Dev Updateโ
envio dev no longer automatically resets the database on incompatible config or schema changes. Use envio dev -r to explicitly allow this.
Envio Start Updateโ
envio start now has a clear role: to run HyperIndex in the production environment. Use envio dev for local development to enable debugging with Dev Console.
Optimized envio codegenโ
envio codegen is now near-instant. We no longer run pnpm i for the generated package, and we no longer recompile ReScript every time you change config.yaml or schema.graphql. The output is also a lot quieter.
envio skills update Commandโ
Pull the latest Claude/Cursor skills into your project so agent-driven development stays in sync with the latest HyperIndex APIs:
pnpx envio skills update
envio config view Command (Experimental)โ
Inspect your fully resolved indexer configuration as JSON โ useful for debugging configuration issues and for tooling that needs to consume the resolved config:
pnpx envio config view
Improved TypeScript Error Messagesโ
When generated types are missing, the TypeScript error now explicitly suggests running envio codegen instead of leaving you to puzzle out the cause.
Smaller envio Package (-88MB)โ
By eliminating dynamically generated ReScript code, we no longer need to ship or run a ReScript compiler at runtime. The published npm package shrank from 141MB to 53MB.
No Hard pnpm Requirementโ
Internal use of pnpm is gone. The generated package no longer has its own dependency tree, so HyperIndex works with whichever package manager you prefer.
Bun Supportโ
Run HyperIndex on Bun:
bun --bun envio dev
Choose Your Package Manager on envio initโ
envio init now accepts --package-manager=pnpm|npm|bun|yarn so you can scaffold projects without committing to pnpm.
Better Tuples Developer Experienceโ
Solidity struct components used to be generated as positional tuples in handler params, which made handler code awkward. They are now generated as objects with named fields:
struct CreateEventCommon {
address funder;
address sender;
address recipient;
Lockup.CreateAmounts amounts;
IERC20 token;
bool cancelable;
bool transferable;
Lockup.Timestamps timestamps;
string shape;
address broker;
}
event CreateLockupTranchedStream(
uint256 indexed streamId,
Lockup.CreateEventCommon commonParams,
LockupTranched.Tranche[] tranches
);
// Before
event.params.commonParams[5];
event.params.commonParams[3][0];
// After
event.params.commonParams.cancelable;
event.params.commonParams.amounts.deposit;
Improved Multichain Backfillโ
For large multichain indexers, HyperIndex now throttles chains that have already reached the head so they don't compete for resources while the rest finish backfilling. Once every chain has caught up, throttling is lifted and all chains continue indexing equally.
Toolchain Upgradesโ
- ReScript upgraded from v11 to v12 (internally and in
envio inittemplates) - TypeScript upgraded from v5 to v6 (internally and in
envio inittemplates)
2x Cheaper and 2.5x Faster (v3.1)โ
HyperIndex now requires up to 2x fewer HyperSync queries during backfill and is 2.5x faster in many indexing cases. If you had data fetching as a bottleneck, this comes for free on upgrade.
Descriptions on Entities, Fields, and Relationships (v3.1)โ
You can now document your entities, fields, and relationships directly in schema.graphql using string descriptions. These are exposed through the GraphQL API and appear in introspection:
"""
A token transfer between two accounts
"""
type Transfer {
id: ID!
"The address the tokens were sent from"
from: String!
"The address the tokens were sent to"
to: String!
"The amount transferred, in wei"
value: BigInt!
}
Only string descriptions ("""...""" or "...") are exposed. Hash (#) comments are ignored by the GraphQL parser and do not appear in introspection.
Skip Chains From Indexing (v3.1)โ
A new skip field in config.yaml lets you exclude a specific chain from indexing and database migrations without removing it from your config:
chains:
- id: 1
start_block: 0
contracts: # ...
- id: 137
skip: true
start_block: 0
contracts: # ...
Improved Agentic Indexer Development (v3.1)โ
New CLI subcommands make it easier to build indexers with AI agents:
envio tools search-docs # Search the HyperIndex documentation
envio tools fetch-docs # Fetch documentation from a URL
envio metrics runtime # Fetch runtime metrics of a local indexer
The skills shipped by envio init and envio skills update were also cleaned up.
Rate-Limit Info in TUI and Logs (v3.1)โ
HyperSync rate-limit handling was improved, and rate-limit information is now surfaced in the TUI and logs so you can see when you're being throttled.
Filter by Multiple Fields with getWhere (v3.2)โ
getWhere now supports filtering by multiple fields simultaneously in a single call:
await context.Account.getWhere({
id: { _eq: "0x123..." },
balance: { _gte: 1_000_000n, _lte: 10_000_000n },
});
Single _eq or _in filters were also optimized to reduce database round trips.
Default Storage (v3.2)โ
When running with multiple storage backends, you can now mark a storage as default so entities are automatically assigned to it without needing a @storage attribute on every entity:
storage:
postgres:
default: true
clickhouse:
default: true
Configurable Column Name Format (v3.2)โ
You can configure HyperIndex to automatically convert database column names to snake_case while keeping the original names in GraphQL and your handler types:
storage:
postgres:
column_name_format: snake_case
Fixesโ
- Fixed an issue where the indexer stops progressing without any error (PostgreSQL client update)
- Fixed checksum for addresses returned by RPC in lowercase
- Fixed incorrect validation of transactions
tofield returned by RPC - Fixed OOM error on RPC request crashing loop
- Fixed an edge case where a multichain indexer could freeze during a rollback on reorg (also backported to v2.32.10)
- Fixed external Postgres database support via
ENVIO_PG_HOST - Fixed
S.nullableschema type to beT | nullinstead ofT | undefined
Release Notesโ
For detailed release notes, see:
- v3.2.0
- v3.1.0
- v3.0.0
- v3.0.0-rc.1
- v3.0.0-rc.0
- v3.0.0-alpha.24
- v3.0.0-alpha.23
- v3.0.0-alpha.22
- v3.0.0-alpha.21
- v3.0.0-alpha.20
- v3.0.0-alpha.19
- v3.0.0-alpha.18
- v3.0.0-alpha.17
- v3.0.0-alpha.16
- v3.0.0-alpha.15
- v3.0.0-alpha.14
- v3.0.0-alpha.13
- v3.0.0-alpha.12
- v3.0.0-alpha.11
- v3.0.0-alpha.10
- v3.0.0-alpha.9
- v3.0.0-alpha.8
- v3.0.0-alpha.7
- v3.0.0-alpha.6
- v3.0.0-alpha.5
- v3.0.0-alpha.4
- v3.0.0-alpha.3
- v3.0.0-alpha.2
- v3.0.0-alpha.1
- v3.0.0-alpha.0
HyperIndex Performance Benchmarksโ
File: benchmarks.md
Overviewโ
HyperIndex delivers industry-leading performance for blockchain data indexing. Independent benchmarks have consistently shown Envio's HyperIndex to be the fastest blockchain indexing solution available, with dramatic performance advantages over competitive offerings.
Recent Independent Benchmarksโ
The most comprehensive and up-to-date benchmarks were conducted by Sentio in April 2025 and are available in the sentio-benchmark repository. These benchmarks compare Envio's HyperIndex against other popular blockchain indexers across multiple real-world scenarios:
Key Performance Highlightsโ
| Case | Description | Envio | Nearest Competitor | The Graph | Ponder |
|---|---|---|---|---|---|
| LBTC Token Transfers | Event handling, No RPC calls, Write-only | 3m | 8m - 2.6x slower (Sentio) | 3h9m - 3780x slower | 1h40m - 2000x slower |
| LBTC Token with RPC calls | Event handling, RPC calls, Read-after-write | 1m | 6m - 6x slower (Sentio) | 1h3m - 63x slower | 45m - 45x slower |
| Ethereum Block Processing | 100K blocks with Metadata extraction | 7.9s | 1m - 7.5x slower (Subsquid) | 10m - 75x slower | 33m - 250x slower |
| Ethereum Transaction Gas Usage | Transaction handling, Gas calculations | 1m 26s | 7m - 4.8x slower (Subsquid) | N/A | 33m - 23x slower |
| Uniswap V2 Swap Trace Analysis | Transaction trace handling, Swap decoding | 41s | 2m - 3x slower (Subsquid) | 8m - 11x slower | N/A |
| Uniswap V2 Factory | Event handling, Pair and swap analysis | 8s | 2m - 15x slower (Subsquid) | 19m - 142x slower | 21m - 157x slower |
The independent benchmark results demonstrate that HyperIndex consistently outperforms all competitors across every tested scenario. This includes the most realistic real-world indexing scenario LBTC Token with RPC calls - where HyperIndex was up to 6x faster than the nearest competitor and over 63x faster than The Graph.
For a wider, benchmark-backed comparison across the rest of the category, see Best Blockchain Indexers in 2026, which covers Envio, The Graph, Goldsky, SubQuery, Subsquid, Ormi, and Ponder side by side.
Historical Benchmarking Resultsโ
Our internal benchmarking from October 2023 showed similar performance advantages. When indexing the Uniswap V3 ETH-USDC pool contract on Ethereum Mainnet, HyperIndex achieved:
- 2.1x faster indexing than the nearest competitor
- Over 100x faster indexing than some popular alternatives
You can read the full details in our Indexer Benchmarking Results blog post.
Verify For Yourselfโ
We encourage developers to run their own benchmarks. You can also use the templates provided in the Open Indexer Benchmark repository.
How to Migrate Using AIโ
File: migrate-with-ai.md
HyperIndex v3 includes built-in Claude skills that guide AI programming assistants through the full subgraph migration process, from understanding your existing logic to converting handlers and running quality checks. This is the recommended way to migrate complex subgraphs.
Prerequisitesโ
- An AI programming assistant (Cursor or Claude Code)
- pnpm installed
- HyperIndex v3 (Claude skills are available in v3)
Step 1: Initialize a Boilerplate HyperIndex Indexerโ
Create a new HyperIndex indexer that indexes the same contracts and events as the subgraph you are migrating. Run the following in a new directory:
pnpx envio init
Follow the CLI prompts to set up the boilerplate indexer with the same contracts and events as your existing subgraph.
The Claude skills are only available in HyperIndex v3. See the v3 migration guide for current install guidance.
Step 2: Set Up a Monorepo Structureโ
Create a parent directory that contains both your new HyperIndex boilerplate indexer and the existing subgraph repo you want to migrate:
my-migration/
โโโ my-subgraph/ # Your existing subgraph repo
โโโ my-hyperindex-indexer/ # The boilerplate HyperIndex indexer from Step 1
This structure gives your assistant visibility into both projects so it can read and understand your subgraph logic while writing the HyperIndex implementation.
Step 3: Run Your AI Programming Assistantโ
Open the monorepo root with your AI programming assistant running there (for example, run Claude Code in the monorepo root or open the monorepo in Cursor). Put your assistant in plan mode first, then provide a prompt like the following (replace the repo names with your own):
This monorepo contains two indexers:
- `my-subgraph/` โ an existing Graph Protocol subgraph indexer (source of truth)
- `my-hyperindex-indexer/` โ a HyperIndex boilerplate scaffolded from the same
contracts (migration target)
Migrate the subgraph indexer to a fully working HyperIndex indexer.
Follow these phases in order:
Phase 1 โ Plan
- Produce a migration plan mapping each subgraph component to its HyperIndex
equivalent.
- Flag anything that has no direct equivalent and propose a workaround.
- Do NOT write code yet.
Phase 2 โ Implement
- Migrate the entire subgraph following the plan and skill guides.
- Process one handler file at a time.
- After each file, run `pnpm envio codegen` to validate, and verify it against
the migration checklist before moving on.
Phase 3 โ Verify
- Walk through every checklist item from the migration skill and confirm it
passes.
- Run any available build or type check commands.
- List any items you could not complete and why.
- Only modify files in `my-hyperindex-indexer/`. Do not change the subgraph repo.
- Preserve all entity fields and event mappings from the subgraph.
- Do not skip or summarize checklist items โ execute every one.
- If you are uncertain about a migration decision, pause and ask me.
- After migration, run
pnpm devto verify the indexer runs correctly - Use the Indexer Migration Validator to compare outputs between your subgraph and the new HyperIndex indexer
Manual Migrationโ
For a detailed manual migration guide covering the step by step conversion of subgraph.yaml, schema, and event handlers, see Migrate from The Graph.
Migrate from The Graph to Envioโ
File: migration-guide.md
Please reach out to our team on Discord for personalized migration assistance.
This page covers migrating from The Graph to Envio, with all examples shown in current HyperIndex V3 syntax (indexer.onEvent(...), chains:). If instead you are upgrading an existing HyperIndex project from V2 to V3, follow the Migrate to V3 guide.
Introductionโ
Migrating your existing subgraph to Envio's HyperIndex is designed to be a developer-friendly process. HyperIndex draws strong inspiration from The Graphโs subgraph architecture, which makes the migration simple, especially with the help of coding assistants like Cursor and AI tools (don't forget to use our ai friendly docs).
The process is simple but requires a good understanding of the underlying concepts. If you are new to HyperIndex, we recommend starting with the Quickstart guide.
If you want an assistant-led workflow, see How to Migrate Using AI for a guided process that works in both Cursor and Claude Code.
Why Migrate to HyperIndex?โ
- Superior Performance: Up to 100x faster indexing speeds
- Lower Costs: Reduced infrastructure requirements and operational expenses
- Better Developer Experience: Simplified configuration and deployment
- Advanced Features: Access to capabilities not available in other indexing solutions
- Seamless Integration: Easy integration with existing GraphQL APIs and applications
If you are still deciding, our comparison of the best blockchain indexers in 2026 covers HyperIndex against The Graph, Goldsky, SubQuery, Subsquid, Ormi, and Ponder with side-by-side benchmarks.
Subgraph to HyperIndex Migration Overviewโ
Migration consists of three major steps:
- Subgraph.yaml migration
- Schema migration - near copy paste
- Event handler migration
At any point in the migration run
pnpm envio codegen
to verify the config.yaml and schema.graphql files are valid.
or run
pnpm dev
to verify the indexer is running and indexing correctly.
0.5 Use pnpx envio init to generate a boilerplateโ
As a first step, we recommend using pnpx envio init to generate a boilerplate for your project. This will handle the creation of the config.yaml file and a basic schema.graphql file with generic handler functions.
1. subgraph.yaml โ config.yamlโ
pnpx envio init will generate this for you. It's a simple configuration file conversion. Effectively specifying which contracts to index, which networks to index (multiple networks can be specified with envio) and which events from those contracts to index.
Take the following conversion as an example, where the subgraph.yaml file is converted to config.yaml the below comparisons is for the Uniswap v4 pool manager subgraph.
The Graph - subgraph.yaml
specVersion: 0.0.4
description: Uniswap is a decentralized protocol for automated token exchange on Ethereum.
repository: https://github.com/Uniswap/v4-subgraph
schema:
file: ./schema.graphql
features:
- nonFatalErrors
- grafting
- kind: ethereum/contract
name: PositionManager
network: mainnet
source:
abi: PositionManager
address: "0xbD216513d74C8cf14cf4747E6AaA6420FF64ee9e"
startBlock: 21689089
mapping:
kind: ethereum/events
apiVersion: 0.0.7
language: wasm/assemblyscript
file: ./src/mappings/index.ts
entities:
- Position
abis:
- name: PositionManager
file: ./abis/PositionManager.json
eventHandlers:
- event: Subscription(indexed uint256,indexed address)
handler: handleSubscription
- event: Unsubscription(indexed uint256,indexed address)
handler: handleUnsubscription
- event: Transfer(indexed address,indexed address,indexed uint256)
handler: handleTransfer
HyperIndex - config.yaml
# yaml-language-server: $schema=./node_modules/envio/evm.schema.json
name: uni-v4-indexer
chains:
- id: 1
start_block: 21689089
contracts:
- name: PositionManager
address: "0xbD216513d74C8cf14cf4747E6AaA6420FF64ee9e"
events:
- event: Subscription(uint256 indexed tokenId, address indexed subscriber)
- event: Unsubscription(uint256 indexed tokenId, address indexed subscriber)
- event: Transfer(address indexed from, address indexed to, uint256 indexed id)
For any potential hurdles, please refer to the Configuration File documentation.
2. Schema migrationโ
copy & paste the schema from the subgraph to the HyperIndex config file.
Small nuance differences:
- You can remove the
@entitydirective - Enums
- BigDecimals
3. Event handler migrationโ
This consists of two parts
- Converting assemblyscript to typescript
- Converting the subgraph syntax to HyperIndex syntax
3.1 Converting Assemblyscript to Typescriptโ
The subgraph uses assemblyscript to write event handlers. The HyperIndex syntax is usually in typescript. Since assemblyscript is a subset of typescript, it's quite simple to copy and paste the code, especially so for pure functions.
3.2 Converting the subgraph syntax to HyperIndex syntaxโ
There are some subtle differences in the syntax of the subgraph and HyperIndex. Including but not limited to the following:
- Replace Entity.save() with context.Entity.set()
- Convert to async handler functions
- Use
awaitfor loading entitiesconst x = await context.Entity.get(id) - Use dynamic contract registration to register contracts
The below code snippets can give you a basic idea of what this difference might look like.
The Graph - eventHandler.ts
export function handleSubscription(event: SubscriptionEvent): void {
const subscription = new Subscribe(event.transaction.hash + event.logIndex);
subscription.tokenId = event.params.tokenId;
subscription.address = event.params.subscriber.toHexString();
subscription.logIndex = event.logIndex;
subscription.blockNumber = event.block.number;
subscription.position = event.params.tokenId;
subscription.save();
}
HyperIndex - eventHandler.ts
indexer.onEvent(
{ contract: "PoolManager", event: "Subscription" },
async ({ event, context }) => {
const entity = {
id: event.transaction.hash + event.logIndex,
tokenId: event.params.tokenId,
address: event.params.subscriber,
blockNumber: event.block.number,
logIndex: event.logIndex,
position: event.params.tokenId,
};
context.Subscription.set(entity);
},
);
Extra tipsโ
HyperIndex is a powerful tool that can be used to index any contract. There are some features that are especially powerful that go above subgraph implementations and so in some cases you may want to optimise your migration to HyperIndex further to take advantage of these features. Here are some useful tips:
- Use
field_selectionto opt into optional transaction and block fields (e.g.hash,status,gasUsed) that are not included by default, see Transaction receipts for a migration-focused example and the field selection docs for the full list. - Multichain indexing in V3 always runs in unordered mode, which is the most common need and provides better performance โ see Multichain Indexing. (In V2 this required setting
unordered_multichain_mode: true; in V3 there is no opt-in, and the V2multichain: orderedmode has been removed.) - Use wildcard indexing to index by event signatures rather than by contract address.
- HyperIndex uses the standard GraphQL query language, whereas TheGraph uses a custom GraphQL syntax. You can read about the differences and how to convert queries in our Query Conversion Guide. We also provide a query converter tool for backwards compatibility with existing TheGraph queries.
- Preload Optimization is always on in V3 and speeds up historical sync by batching the entity reads in your handlers and running external calls in parallel. You can read more about it here.
- HyperIndex is very flexible and can be used to index offchain data too or send messages to a queue etc for fetching external data, you can further optimise the fetching by using the effects api
Transaction receiptsโ
In The Graph, you opt into receipt data per-handler with receipt: true in subgraph.yaml:
eventHandlers:
- event: Transfer(indexed address,indexed address,indexed uint256)
handler: handleTransfer
receipt: true
This makes event.receipt available inside the handler with fields like status, gasUsed, and logs.
In HyperIndex, receipt-level fields are part of transaction_fields and must be requested via field_selection in config.yaml. There is no separate receipt object โ the fields are accessed directly on event.transaction:
field_selection:
transaction_fields:
- hash
- status # 1 = success, 0 = reverted
- gasUsed
- cumulativeGasUsed
- contractAddress # non-null for contract-creation transactions
- logsBloom
indexer.onEvent(
{ contract: "MyContract", event: "Transfer" },
async ({ event, context }) => {
const { status, gasUsed } = event.transaction;
// ...
},
);
See the full list of available transaction_fields in the Configuration File docs.
Validating Your Migrationโ
After completing your migration, it's important to verify that your HyperIndex indexer produces the same data as your original subgraph. Use the Indexer Migration Validator CLI tool to compare results between both endpoints and identify any discrepancies. The tool automatically generates entity configs from your GraphQL schema and provides detailed field-level analysis of differences.
Share Your Learningsโ
If you discover helpful tips during your migration, we'd love contributions! Open a PR to this guide and help future developers.
Getting Helpโ
Join Our Discord: The fastest way to get personalized help is through our Discord community.
Migrate from Ponder to HyperIndexโ
File: migrate-from-ponder.md
Need help? Reach out on Discord for personalized migration assistance.
Migrating from Ponder to HyperIndex is straightforward โ both frameworks use TypeScript, index EVM events, and expose a GraphQL API. The key differences are the config format, schema syntax, and entity operation API.
If you are new to HyperIndex, start with the Quickstart guide first. If you are new to the category entirely, see what a blockchain indexer is for the wider context.
For an assistant-led workflow, see How to Migrate Using AI, which includes a shared process for Cursor and Claude Code.
Why Migrate to HyperIndex?โ
- Up to 158x faster historical sync via HyperSync
- Multichain by default โ index any number of chains in one config
- Same language โ your TypeScript logic transfers directly
Migration Overviewโ
Migration has three steps:
ponder.config.tsโconfig.yamlponder.schema.tsโschema.graphql- Event handlers โ adapt syntax and entity operations
At any point, run:
pnpm envio codegen # validate config + schema, regenerate types
pnpm dev # run the indexer locally
Step 1: ponder.config.ts โ config.yamlโ
Ponder
export default createConfig({
chains: {
mainnet: { id: 1, rpc: process.env.PONDER_RPC_URL_1 },
},
contracts: {
MyToken: {
abi: myTokenAbi,
chain: "mainnet",
address: "0xabc...",
startBlock: 18000000,
},
},
});
HyperIndex (v3)
# yaml-language-server: $schema=./node_modules/envio/evm.schema.json
name: my-indexer
contracts:
- name: MyToken
abi_file_path: ./abis/MyToken.json
events:
- event: Transfer
- event: Approval
chains:
- id: 1
start_block: 0
contracts:
- name: MyToken
address:
- 0xabc...
start_block: 18000000
v2 note: HyperIndex v2 uses
networksinstead ofchains. See the v2โv3 migration guide.
Key differences:
| Concept | Ponder | HyperIndex |
|---|---|---|
| Config format | ponder.config.ts (TypeScript) | config.yaml (YAML) |
| Chain reference | Named + viem object | Numeric chain ID |
| RPC URL | In config | RPC_URL_ env var |
| ABI source | TypeScript import | JSON file (abi_file_path) |
| Events to index | Inferred from handlers | Explicit events: list |
| Handler file | Inferred | Explicit handler: per contract |
Convert your ABI: Ponder uses TypeScript ABI exports (as const). HyperIndex needs a plain JSON file in abis/. Strip the export const ... = wrapper and as const and save as .json.
Field selection โ accessing transaction and block fieldsโ
By default, only a minimal set of fields is available on event.transaction and event.block. Fields like event.transaction.hash are undefined unless explicitly requested.
events:
- event: Transfer
field_selection:
transaction_fields:
- hash
Or declare once at the top level to apply to all events:
name: my-indexer
field_selection:
transaction_fields:
- hash
contracts:
# ...
See the full list of available fields in the Configuration File docs.
Migrate From Alchemyโ
File: migrate-from-alchemy.md
Note: Alchemy subgraphs sunset on Dec 8th, 2025. Envio is offering affected Alchemy users 2 months of free hosting on Envio, along with full white-glove migration support to help projects move over smoothly.
For more info on how you can start your free trial or book migration support, visit this page to learn more.
Migrating Alchemy subgraphs to Envioโs HyperIndex is a simple and developer-friendly process. Alchemy subgraphs follow The Graphโs model and HyperIndex uses a very similar structure, so most of your existing setup can carry over cleanly.
If you're familiar with The Graphโs libraries, the migration process should be straightforward. You can also utilize tools like Cursor to speed things up. If you are new to HyperIndex, we strongly recommend starting with our Quickstart guide before you begin your migration from Alchemy. If you are new to the category entirely, see what a blockchain indexer is for the wider context.
Why Migrate to Envioโs HyperIndex?โ
- High-Speed Performance: 142x faster than subgraphs
- Lower Costs: Reduced infrastructure requirements and operational expenses
- Better Developer Experience: Simplified configuration and deployment
- Multichain Native: Index data across multiple EVM chains through a single HyperIndex project
- Local Development: Run your indexers locally for fast iteration and easier debugging
- White Glove Migration Support: Get direct support from the Envio team for a smoother migration.
- GitOps Ready Deployments: Link your GitHub repo and manage multiple deployments in a clean unified workflow
- Advanced Features: Access to features like external calls and block handlers
- Seamless Integration: Easily integrate existing GraphQL APIs and applications
How to Migrate from Alchemy to Envio in 4 easy stepsโ
This Migration consists of 4 major steps:
- Create a HyperIndex Project
- subgraph.yaml Migration to config.yaml
- schema.graphql Migration
- Event Handler Migration
Create a HyperIndex Projectโ
Start by spinning up a basic HyperIndex project with this command:
pnpx envio init template --name alchemy-migration --directory alchemy-migration --template greeter --api-token "YOUR_ENVIO_API_KEY"
Once the project is created, drop your API key into the .env file and youโre good to go.
subgraph.yaml Migration to config.yamlโ
In HyperIndex, all project configuration lives in config.yaml. This is where you define contract addresses, the networks you want to index, and the specific events you want to track from those contracts.
Below is an example showing how a Uniswap V4 subgraph.yaml maps to a HyperIndex config.yaml in a real migration.
The Graph - subgraph.yaml
specVersion: 0.0.4
description: Uniswap is a decentralized protocol for automated token exchange on Ethereum.
repository: https://github.com/Uniswap/v4-subgraph
schema:
file: ./schema.graphql
features:
- nonFatalErrors
- grafting
- kind: ethereum/contract
name: PositionManager
network: mainnet
source:
abi: PositionManager
address: "0xbD216513d74C8cf14cf4747E6AaA6420FF64ee9e"
startBlock: 21689089
mapping:
kind: ethereum/events
apiVersion: 0.0.7
language: wasm/assemblyscript
file: ./src/mappings/index.ts
entities:
- Position
abis:
- name: PositionManager
file: ./abis/PositionManager.json
eventHandlers:
- event: Subscription(indexed uint256,indexed address)
handler: handleSubscription
- event: Unsubscription(indexed uint256,indexed address)
handler: handleUnsubscription
- event: Transfer(indexed address,indexed address,indexed uint256)
handler: handleTransfer
HyperIndex - config.yaml
# yaml-language-server: $schema=./node_modules/envio/evm.schema.json
name: uni-v4-indexer
chains:
- id: 1
start_block: 21689089
contracts:
- name: PositionManager
address: "0xbD216513d74C8cf14cf4747E6AaA6420FF64ee9e"
events:
- event: Subscription(uint256 indexed tokenId, address indexed subscriber)
- event: Unsubscription(uint256 indexed tokenId, address indexed subscriber)
- event: Transfer(address indexed from, address indexed to, uint256 indexed id)
If you hit any issues, check the Configuration File docs or reach out to our team in Discord.
schema.graphql Migrationโ
This step is simple. You keep the entire file as is, with one small change: remove all @entity directives from your entities. Everything else stays the same.
Event Handler Migrationโ
This is the final step of the migration which consists of two parts:
- Moving from AssemblyScript to TypeScript
- Updating Subgraph syntax to HyperIndex syntax
AssemblyScript to TypeScriptโ
HyperIndex uses TypeScript instead of AssemblyScript. Since AssemblyScript is a subset of TypeScript, you can simply copy most of your code over without worrying about major syntax changes.
Subgraph to HyperIndexโ
The HyperIndex workflow is very similar to Subgraphs, but there are a few important differences to keep in mind:
- Replace
ENTITY.save()withcontext.ENTITY.set(VALUES) - Handlers need to be async
- Use
awaitwhen loading entities
As you start using HyperIndex, youโll pick up the differences quickly.
Here is a code snippet to give you a sense of what these changes look like in practice.
The Graph - eventHandler.ts
export function handleSubscription(event: SubscriptionEvent): void {
const subscription = new Subscribe(event.transaction.hash + event.logIndex);
subscription.tokenId = event.params.tokenId;
subscription.address = event.params.subscriber.toHexString();
subscription.logIndex = event.logIndex;
subscription.blockNumber = event.block.number;
subscription.position = event.params.tokenId;
subscription.save();
}
HyperIndex - eventHandler.ts
import { indexer } from "envio";
indexer.onEvent(
{ contract: "PoolManager", event: "Subscription" },
async ({ event, context }) => {
const entity = {
id: event.transaction.hash + event.logIndex,
tokenId: event.params.tokenId,
address: event.params.subscriber,
blockNumber: event.block.number,
logIndex: event.logIndex,
position: event.params.tokenId,
};
context.Subscription.set(entity);
},
);
For a few extra tips on migrating from Alchemy to Envio, check out our other migration guide in our docs.
Share Your Learningsโ
If you come across anything useful during your migration, please feel free to contribute. Simply open a PR to this guide and help future developers.
Getting Helpโ
Join our Discord if you need support. It is the fastest way to get direct help from the team and the community.
Migrate to HyperIndex V3โ
File: migrate-to-v3.md
This guide covers every change required to upgrade a HyperIndex V2 project to V3. For new V3 capabilities, see What's New in V3.
Easiest path โ prompt your AI tool (Claude/Cursor/Codex):
Upgrade my indexer to V3 by following the migration instructions step by step https://docs.envio.dev/docs/HyperIndex/migrate-to-v3
Step 0: Prepare on V2 (Recommended)โ
While still on V2:
- Upgrade to
envio@^2.32.6. - Set
preload_handlers: trueinconfig.yaml. - If using loaders, migrate them per Migrating from Loaders.
- Verify with
pnpm dev.
Step 1: Update Node.jsโ
Use Node.js 22+ (24 recommended). Earlier versions are unsupported.
Step 2: Update package.jsonโ
- Add
"type": "module"(required โ without it the project fails to start with ESM errors). - Set
engines.nodeto>=22.0.0. - Update
envioto the latest v3 release. - Remove
optionalDependencies.generatedโ the localgeneratedpackage no longer exists.
{
"type": "module",
"engines": { "node": ">=22.0.0" },
"dependencies": { "envio": "3.0.0" },
"devDependencies": {
"@types/node": "24.12.2",
"typescript": "6.0.3",
"vitest": "4.1.0"
}
}
If you used ts-node for the start script, replace it with "start": "envio start".
Test runnerโ
Option A โ Vitest (recommended).
pnpm remove ts-mocha ts-node mocha chai @types/mocha @types/chai
pnpm add -D vitest@4.0.16
Set "test": "vitest run", then move test/Test.ts โ src/indexer.test.ts and update imports:
// Before (mocha/chai)
// After (vitest)
Option B โ Keep Mocha. Replace ts-mocha/ts-node with tsx:
pnpm remove ts-mocha ts-node
pnpm add -D tsx@4.21.0
{
"scripts": {
"mocha": "tsc --noEmit && NODE_OPTIONS='--no-warnings --import tsx' mocha --exit test/**/*.ts"
}
}
Step 3: Update tsconfig.jsonโ
Update for ESM (copy-paste the file as-is, comments included):
{
/* For details: https://www.totaltypescript.com/tsconfig-cheat-sheet */
"compilerOptions": {
/* Base Options: */
"esModuleInterop": true,
"skipLibCheck": true,
"target": "es2022",
"allowJs": true,
"resolveJsonModule": true,
"moduleDetection": "force",
"isolatedModules": true,
"verbatimModuleSyntax": true,
/* Strictness */
"strict": true,
"noUncheckedIndexedAccess": true,
"noImplicitOverride": true,
/* For running Envio: */
"module": "ESNext",
"moduleResolution": "bundler",
"noEmit": true,
/* Code doesn't run in the DOM: */
"lib": ["es2022"],
"types": ["node"]
}
}
verbatimModuleSyntax and noUncheckedIndexedAccess are optional extra strictness โ disable them to simplify migration.
Step 4: Update config.yamlโ
Renames:
networksโchainsconfirmed_block_thresholdโmax_reorg_depthrpc_configโrpc(now supports multiple URLs,for: sync | realtime | fallback, and WebSocket config)
Remove if present:
unordered_multichain_modeand anymultichain: orderedโ unordered is the only mode in V3.loaders,preload_handlersโ Preload Optimization is always enabled.preRegisterDynamicContractsโ no longer needed.event_decoderโ the Rust decoder is the only implementation.outputโ types always emitted to.envio/.
Env var โ config: replace the MAX_BATCH_SIZE env var with full_batch_size: 5000.
Optional (recommended): move handler files to src/handlers/ and drop the explicit handler paths (the handler field still works).
Step 5: Update Environment Variablesโ
Add โ if using HyperSync (the default), set ENVIO_API_TOKEN (get a free token at envio.dev/app/api-tokens).
Remove:
UNSTABLE__TEMP_UNORDERED_HEAD_MODEUNORDERED_MULTICHAIN_MODEMAX_BATCH_SIZE(usefull_batch_sizeinconfig.yaml)ENVIO_INDEXING_BLOCK_LAG(use per-chainblock_lag)
Rename:
TUI_OFF=trueโENVIO_TUI=false(TUI also auto-disabled in CI and under AI agents)ENVIO_PG_PUBLIC_SCHEMAโENVIO_PG_SCHEMA(old name supported until v4)
Step 6: Update Handler Codeโ
Contract-specific exports are removed. Register handlers through the unified indexer from the envio package, which replaces generated.
Event handlersโ
// Before
ERC20.Transfer.handler(
async ({ event, context }) => {},
{
wildcard: true,
eventFilters: ({ chainId }) => [
{ from: ZERO_ADDRESS, to: WHITELIST[chainId] },
],
}
);
// After
indexer.onEvent(
{
contract: "ERC20",
event: "Transfer",
wildcard: true,
where: ({ chain }) => ({
params: [{ from: ZERO_ADDRESS, to: WHITELIST[chain.id] }],
}),
},
async ({ event, context }) => {},
);
eventFiltersโwhere. Callback receives{ chain }(not{ chainId }) and returnsfalse,true, or{ params: [...], block?: { number: { _gte, _lte, _every } } }.- The top-level array shorthand is gone โ wrap it in
{ params: [...] }.
Filtering by the contract's own addresses โ V2's eventFilters addresses argument becomes chain..addresses (kept in sync with context.chain..add(...)):
// Before
Safe.Transfer.handler(async ({ event, context }) => {}, {
wildcard: true,
eventFilters: ({ addresses }) => [{ from: addresses }, { to: addresses }],
});
// After
indexer.onEvent(
{
contract: "Safe",
event: "Transfer",
wildcard: true,
where: ({ chain }) => ({
params: [{ from: chain.Safe.addresses }, { to: chain.Safe.addresses }],
}),
},
async ({ event, context }) => {},
);
Dynamic contract registrationโ
// Before
UniV3.PoolFactory.contractRegister(async ({ event, context }) => {
context.addPool(event.params.poolAddress);
});
// After
indexer.contractRegister(
{ contract: "UniV3", event: "PoolFactory" },
async ({ event, context }) => {
context.chain.Pool.add(event.params.poolAddress);
},
);
context.add(addr) โ context.chain..add(addr).
Block handlersโ
Behavior change. V2's onBlock ran on one chain (its chain option) with top-level interval/startBlock/endBlock. V3's indexer.onBlock runs on every chain by default. To restore V2's single-chain + range + interval behavior, pass a where callback that returns false for unwanted chains and { block: { number: { _gte, _lte, _every } } } for the range/interval.
// Before โ only chain 1, every 100 blocks, fixed range
onBlock(
{ name: "Ranges", chain: 1, startBlock: 20_000_000, endBlock: 22_000_000, interval: 100 },
async ({ block, context }) => {},
);
// After
indexer.onBlock(
{
name: "Ranges",
where: ({ chain }) => {
if (chain.id !== 1) return false;
return { block: { number: { _gte: 20_000_000, _lte: 22_000_000, _every: 100 } } };
},
},
async ({ block, context }) => {},
);
To run on every chain (the new default), omit where. Inside the handler, block.chainId โ context.chain.id.
getWhere APIโ
Switch to GraphQL-style filter syntax (new operators: _gte, _lte, _in):
// Before
await context.Transfer.getWhere.from.eq("0x123...");
await context.Transfer.getWhere.value.gt(1000n);
// After
await context.Transfer.getWhere({ from: { _eq: "0x123..." } });
await context.Transfer.getWhere({ value: { _gt: 1000n } });
Rename and removal cheat sheetโ
| V2 (removed) | V3 |
|---|---|
Contract.Event.handler(...) | indexer.onEvent({ contract, event, ...options }, handler) |
Contract.Event.contractRegister(...) | indexer.contractRegister({ contract, event }, handler) |
onBlock({ chain, ... }, handler) | indexer.onBlock({ name, where? }, handler) |
context.add(addr) | context.chain..add(addr) |
eventFilters option | where callback returning { params: [...] } |
experimental_createEffect | createEffect |
block.chainId (in block handlers) | context.chain.id |
transaction.kind | transaction.type |
transaction.chainId | context.chain.id or event.chainId |
chain type | ChainId (now a union type) |
getGeneratedByChainId(...) | indexer.chains[chainId] |
Entity.getWhere.field.eq(value) | Entity.getWhere({ field: { _eq: value } }) |
Entity.getWhere.field.gt(value) | Entity.getWhere({ field: { _gt: value } }) |
Entity.getWhere.field.lt(value) | Entity.getWhere({ field: { _lt: value } }) |
Lowercased entity types (e.g. transfer) | Capitalized (Transfer) |
ERC20_Transfer_eventLog | EvmEvent |
ERC20_Transfer_block | EvmEvent["block"] |
MyEnum (direct export) | Enum |
MyEntity (direct export) | Entity (preferred; direct still exported) |
Other type changes: Address is now `0x${string}` (was string); entity array fields are readonly; S.nullable returns T | null (was T | undefined); the internal ContractType enum was removed.
Step 7: Remove generatedโ
The generated package is no longer needed โ remove it. Import everything from "envio" instead. This works via envio-env.d.ts, which is linked automatically (no tsconfig.json change needed).
Step 8: Update Testsโ
MockDb is removed. Use createTestIndexer() with simulate.
-import { TestHelpers, type User } from "generated";
-const { MockDb, Greeter, Addresses } = TestHelpers;
+import { createTestIndexer, type User, TestHelpers } from "envio";
+const { Addresses } = TestHelpers;
it("A NewGreeting event creates a User entity", async (t) => {
- const mockDbInitial = MockDb.createMockDb();
+ const indexer = createTestIndexer();
const userAddress = Addresses.defaultAddress;
const greeting = "Hi there";
- const mockNewGreetingEvent = Greeter.NewGreeting.createMockEvent({
- greeting: greeting,
- user: userAddress,
- });
-
- const updatedMockDb = await Greeter.NewGreeting.processEvent({
- event: mockNewGreetingEvent,
- mockDb: mockDbInitial,
- });
+ await indexer.process({
+ chains: {
+ 137: {
+ simulate: [
+ { contract: "Greeter", event: "NewGreeting", params: { greeting, user: userAddress } },
+ ],
+ },
+ },
+ });
const expectedUserEntity: User = {
id: userAddress,
latestGreeting: greeting,
numberOfGreetings: 1,
greetings: [greeting],
};
- const actualUserEntity = updatedMockDb.entities.User.get(userAddress);
+ const actualUserEntity = await indexer.User.getOrThrow(userAddress);
t.expect(actualUserEntity).toEqual(expectedUserEntity);
});
Old (MockDb) | New (createTestIndexer) |
|---|---|
MockDb.createMockDb() | createTestIndexer() |
Contract.Event.createMockEvent({...}) | Inline in simulate: [{ contract, event, params }] |
Contract.Event.processEvent({event,mockDb}) | indexer.process({ chains: { id: { simulate } } }) |
mockDb.entities.Entity.get(id) | await indexer.Entity.getOrThrow(id) |
mockDb.entities.Entity.set({...}) | indexer.Entity.set({...}) |
| Manual handler threading & event chaining | Automatic โ pass multiple events in simulate |
Step 9: Update CLI Usageโ
envio devno longer auto-resets the DB โ useenvio dev -r(--restart) if you relied on that.envio startis now production-only; useenvio devfor local development.- Handler file changes no longer trigger codegen on
pnpm dev.
Step 10: Run Codegen and Verifyโ
pnpm envio codegen
pnpm dev
Postgres column type changes (raw_events.event_id: NUMERICโBIGINT, raw_events.serial: SERIALโBIGSERIAL, envio_chains.events_processed: INTEGERโBIGINT, envio_checkpoints.id: INTEGERโBIGINT) apply automatically. The deprecated envio_chains._num_batches_fetched always returns 0.
Step 11: Update Agent Skillsโ
Refresh the bundled agent skills so agent-driven development stays aligned with V3:
pnpx envio skills update
This populates .claude/skills (consumed by Claude, Cursor, and other agentic tooling). Re-run it on each new HyperIndex release.
Getting Helpโ
Issues during migration? Join our Discord community.
Configuration Fileโ
File: Guides/configuration-file.mdx
The config.yaml file defines your indexer's behavior, including which blockchain events to index, contract addresses, which chains to index, and various advanced indexing options. It is a crucial step in configuring your HyperIndex setup.
Whenever you make changes in config.yaml that should affect generated types (e.g. adding events, contracts, or chains), run pnpm codegen to regenerate types and code for your event handlers.
Contracts Definitionโ
The top-level contracts block defines each contract once โ its events and per-event options. Each chain then references these contracts by name and supplies chain-specific values like the on-chain address (see Contracts (per chain)).
contracts:
- name: Greeter
events:
- event: "NewGreeting(address user, string greeting)"
- event: "ClearGreeting(address user)"
chains:
- id: 1
start_block: 0
contracts:
- name: Greeter
address: "0x9D02A17dE4E68545d3a58D3a20BbBE0399E05c9c"
Events Selectionโ
The recommended way to declare events is by their human-readable signature directly under events:
contracts:
- name: Greeter
events:
- event: "NewGreeting(address user, string greeting)"
- event: "ClearGreeting(address user)"
Only the events listed here are indexed. To stop indexing an event, remove its entry.
Custom Event Namesโ
You can assign custom names to events in config.yaml. This is handy when
two events share the same name but have different signatures, or when you want
a more descriptive name in your Envio project.
events:
- event: Assigned(address indexed recipientId, uint256 amount, address token)
- event: Assigned(address indexed recipientId, uint256 amount, address token, address sender)
name: AssignedWithSender
Using an ABI Fileโ
If you'd rather reference a JSON ABI file (e.g. when you have one already and want to pick a subset of events from it), use abi_file_path and refer to events by name:
contracts:
- name: Greeter
abi_file_path: ./abis/greeter.json
events:
- event: NewGreeting # signature comes from the ABI file
Event signatures are still the recommended default โ they keep config.yaml self-contained and easier to review.
Field Selectionโ
To improve indexing performance and reduce credits usage, the block and transaction fields on events contain only a subset of the fields available on the blockchain.
To access fields that are not provided by default, specify them using the field_selection option for your event:
events:
- event: "Assigned(address indexed user, uint256 amount)"
field_selection:
transaction_fields:
- transactionIndex
block_fields:
- timestamp
See all possible options in the Config File Reference or use IDE autocomplete for your help.
Global Field Selectionโ
You can also specify fields globally for all events in the root of the config file:
field_selection:
transaction_fields:
- hash
- gasUsed
block_fields:
- parentHash
Try to use this option sparingly as it can cause redundant Data Source calls and increased credits usage.
Storageโ
How indexed data is persisted โ which backends to use and whether to keep raw event records around.
Storage Backendsโ
By default, HyperIndex writes entities to Postgres. In V3 you can additionally enable ClickHouse as a second storage backend (experimental):
storage:
postgres: true
clickhouse: true
When both backends are enabled, you must route each entity explicitly via the @storage directive in schema.graphql:
# Stored in both Postgres and ClickHouse
type Transfer @storage(postgres: true, clickhouse: true) {
id: ID!
from: String!
to: String!
value: BigInt!
}
# Stored only in ClickHouse
type Snapshot @storage(clickhouse: true) {
id: ID!
blockNumber: BigInt!
}
Alternatively, mark one or both backends as default so every entity is written there automatically, and reach for the @storage directive only when you need to override routing for a specific entity:
storage:
postgres:
default: true
clickhouse:
default: true
The default storage option was added in HyperIndex v3.2.
envio dev automatically spins up a ClickHouse Docker container for local development. For envio start, provide the connection via the environment variables ENVIO_CLICKHOUSE_HOST, ENVIO_CLICKHOUSE_DATABASE, ENVIO_CLICKHOUSE_USERNAME, and ENVIO_CLICKHOUSE_PASSWORD.
Do not run multiple indexers writing to the same ClickHouse database at the same time.
Envio Cloud currently supports ClickHouse on the Dedicated Plan.
Column Name Formatโ
Set column_name_format: snake_case on a backend to store column names in snake_case while keeping the original names in the GraphQL API and your handler types. It works for both Postgres and ClickHouse:
storage:
postgres:
column_name_format: snake_case
clickhouse:
column_name_format: snake_case
Added in HyperIndex v3.2.
Ecosystemโ
ecosystem selects which chain family your indexer targets. EVM is the default and what every example above assumes, but the same config.yaml schema is shared with Fuel and Solana (SVM).
ecosystem: evm # default โ also: fuel, svm
Most projects don't set this field explicitly. Use pnpx envio init fuel or pnpx envio init svm to scaffold non-EVM indexers โ the generated config.yaml will already have ecosystem set correctly. See the Fuel and Solana guides for the ecosystem-specific options.
Advancedโ
In ~95% of cases you don't need to touch any of these โ the defaults are tuned for the common path. Reach for them only when you have a specific reason.
Handler File Pathโ
Handlers are auto-discovered from src/handlers/. Override the directory with the top-level handlers option, or set a per-contract handler path when needed:
handlers: ./src/my-handlers # optional override of src/handlers
contracts:
- name: Greeter
handler: ./src/GreeterHandler.ts # optional per-contract path
Schema File Pathโ
You can customize the path to the schema file using the schema option:
schema: ./path/to/schema.graphql
By default, the schema.graphql is expected to be in the root directory of your project.
Block Lagโ
Set block_lag on a chain to keep the indexer a fixed number of blocks behind the chain head. Defaults to 0.
chains:
- id: 1
start_block: 0
block_lag: 5
Only set block_lag if you understand the trade-off โ it intentionally trades head latency for additional reorg safety.
Rollback on Reorgโ
HyperIndex automatically handles blockchain reorganizations by default. To disable or customize this behavior, set the rollback_on_reorg flag in your config.yaml:
rollback_on_reorg: true # default is true
See detailed configuration options here.
Full Batch Sizeโ
Set full_batch_size to control how many events are processed in a single batch.
full_batch_size: 5000
Raw Events Storageโ
By default, HyperIndex doesn't store raw event data in the database to optimize performance and reduce storage requirements. However, you can enable this feature for debugging purposes or if you need to access the original event data.
To enable storage of raw events, add the following to your config.yaml:
raw_events: true
When enabled, all indexed events will be stored in the raw_events table in the database, which you can view through the Hasura interface. This is particularly useful for:
- Debugging event processing issues
- Verifying that events are being captured correctly
- Creating custom queries against raw blockchain data
Note that enabling this option will increase database storage requirements and may slightly impact indexing performance.
Skip Chainโ
Set skip: true on a chain to exclude it from indexing and migrations without removing it from your config โ handy for temporarily disabling a chain.
chains:
- id: 137
skip: true
start_block: 0
Added in HyperIndex v3.1.
Now your configuration file is set, you're ready to start indexing with HyperIndex!
Schema Fileโ
File: Guides/schema-file.md
The schema.graphql file defines the data model for your HyperIndex indexer. Each entity type defined in this schema corresponds directly to a database table, with your event handlers responsible for creating and updating the records. HyperIndex automatically generates a GraphQL API based on these entity types, allowing easy access to the indexed data.
Scalar Typesโ
Scalar types represent basic data types and map directly to JavaScript, TypeScript, or ReScript types.
| GraphQL Scalar | Description | JavaScript/TypeScript | ReScript |
|---|---|---|---|
ID | Unique identifier | string | string |
String | UTF-8 character sequence | string | string |
Int | Signed 32-bit integer | number | int |
Float | Signed floating-point number | number | float |
Boolean | true or false | boolean | bool |
Bytes | UTF-8 character sequence (hex prefixed 0x) | string | string |
BigInt | Signed integer (int256 in Solidity) | bigint | bigint |
BigDecimal | Arbitrary-size floating-point | BigDecimal (imported) | BigDecimal.t |
Timestamp | Timestamp with timezone | Date | Js.Date.t |
Json | JSON object | Json | Js.Json.t |
Learn more about GraphQL scalars here.
Enum Typesโ
Enums allow fields to accept only a predefined set of values.
Example:
enum AccountType {
ADMIN
USER
}
type User {
id: ID!
balance: Int!
accountType: AccountType!
}
Enums translate to string unions (TypeScript/JavaScript) or polymorphic variants (ReScript):
TypeScript Example:
let user = {
id: event.params.id,
balance: event.params.balance,
accountType: "USER" satisfies Enum, // enum as string
};
ReScript Example:
let user: Types.userEntity = {
id: event.params.id,
balance: event.params.balance,
accountType: #USER, // polymorphic variant
};
Field Indexing (@index)โ
Add an index to a field for optimized queries and loader performance:
type Token {
id: ID!
tokenId: BigInt!
collection: NftCollection!
owner: User! @index
}
- All
idfields and fields referenced via@derivedFromare indexed automatically.
Documenting Entities, Fields, and Relationshipsโ
You can document your entities, fields, and relationships directly in schema.graphql using GraphQL string descriptions. These descriptions are exposed through the generated GraphQL API and appear in introspection, making your API self-documenting.
"""
A token transfer between two accounts
"""
type Transfer {
id: ID!
"The address the tokens were sent from"
from: String!
"The address the tokens were sent to"
to: String!
"The amount transferred, in wei"
value: BigInt!
}
Both single-line ("...") and multi-line ("""...""") descriptions are supported.
Only string descriptions are exposed in introspection. Hash (#) comments are ignored by the GraphQL parser and do not appear in the API. Descriptions on entities, fields, and relationships were added in HyperIndex v3.1.
Best Practicesโ
- Use camelCase for field names (
latestGreeting,numberOfGreetings). - Keep entity and field names clear, descriptive, and intuitive.
Event Handlersโ
File: Guides/event-handlers.mdx
Registrationโ
A handler is a function that receives blockchain data, processes it, and inserts it into the database. You can register handlers in the file defined in the handler field in your config.yaml file. By default this is src/handlers file.
indexer.onEvent(
{ contract: "", event: "" },
async ({ event, context }) => {
// Your logic here
},
);
The envio module exposes the unified indexer value along with types based on your config.yaml and schema.graphql files. Run pnpm codegen whenever you change these files to regenerate the types in .envio/.
Basic Exampleโ
Here's a handler example for the NewGreeting event. It belongs to the Greeter contract from our beginners Greeter Tutorial:
// Handler for the NewGreeting event
indexer.onEvent(
{ contract: "Greeter", event: "NewGreeting" },
async ({ event, context }) => {
const userId = event.params.user; // The id for the User entity
const latestGreeting = event.params.greeting; // The greeting string that was added
const currentUserEntity = await context.User.get(userId); // Optional user entity that may already exist
// Update or create a new User entity
const userEntity: User = currentUserEntity
? {
id: userId,
latestGreeting,
numberOfGreetings: currentUserEntity.numberOfGreetings + 1,
greetings: [...currentUserEntity.greetings, latestGreeting],
}
: {
id: userId,
latestGreeting,
numberOfGreetings: 1,
greetings: [latestGreeting],
};
context.User.set(userEntity); // Set the User entity in the DB
},
);
Preload Optimizationโ
Important! Preload optimization makes your handlers run twice.
Preload optimization is always enabled in HyperIndex V3 โ there is no config flag to toggle it.
This optimization enables HyperIndex to efficiently preload entities used by handlers through batched database queries, while ensuring events are processed synchronously in their original order. When combined with the Effect API for external calls, this feature delivers performance improvements of multiple orders of magnitude compared to other indexing solutions.
Read more in the dedicated guides:
- How Preload Optimization Works
- Double-Run Footgun
- Effect API
Advanced Use Casesโ
HyperIndex provides many features to help you build more powerful and efficient indexers. There's definitely the one for you:
- Handle Factory Contracts with Dynamic Contract Registration (with nested factories support)
- Perform external calls to decide which contract address to register using Async Contract Register
- Index all ERC20 token transfers with Wildcard Indexing
- Use Topic Filtering to ignore irrelevant events
- With multiple filters for single event
- With different filters per chain
- With filter by dynamicly registered contract addresses (eg Index all ERC20 transfers to/from your Contract)
- Access Contract State directly from handlers
- Perform external calls from handlers by following the IPFS Integration guide
Context Objectโ
The handler context provides methods to interact with entities stored in the database.
Retrieving Entitiesโ
Retrieve entities from the database using context.Entity.get where Entity is the name of the entity you want to retrieve, which is defined in your schema.graphql file.
await context.Entity.get(entityId);
It'll return Entity object or undefined if the entity doesn't exist.
Use context.Entity.getOrThrow to conveniently throw an error if the entity doesn't exist:
const pool = await context.Pool.getOrThrow(poolId);
// Will throw: Entity 'Pool' with ID '...' is expected to exist.
// Or you can pass a custom message as a second argument:
const pool = await context.Pool.getOrThrow(
poolId,
`Pool with ID ${poolId} is expected.`
);
Or use context.Entity.getOrCreate to automatically create an entity with default values if it doesn't exist:
const pool = await context.Pool.getOrCreate({
id: poolId,
totalValueLockedETH: 0n,
});
// Which is equivalent to:
let pool = await context.Pool.get(poolId);
if (!pool) {
pool = {
id: poolId,
totalValueLockedETH: 0n,
};
context.Pool.set(pool);
}
Retrieving Entities by Fieldโ
indexer.onEvent(
{ contract: "ERC20", event: "Approval" },
async ({ event, context }) => {
// Find all approvals for this specific owner
const currentOwnerApprovals = await context.Approval.getWhere({
owner_id: { _eq: event.params.owner },
});
// Process all the owner's approvals efficiently
for (const approval of currentOwnerApprovals) {
// Process each approval
}
},
);
Beyond _eq, you can filter by value using comparison operators like _gt, _gte, _lt, _lte, and _in.
To narrow results further, combine several conditions in a single call โ they all have to match (AND):
// Find accounts matching a specific id AND a balance within a range
const accounts = await context.Account.getWhere({
id: { _eq: event.params.account },
balance: { _gte: 1_000_000n, _lte: 10_000_000n },
});
Multi-field filtering with getWhere was added in HyperIndex v3.2.
Important:
-
Preload Optimization is always enabled in V3 and powers
getWhere. See How Preload Optimization Works. -
Works with any field that:
- Is used in a relationship with the
@derivedFromdirective - Has an
@indexdirective
- Is used in a relationship with the
-
Potential Memory Issues: Very large
getWherequeries might cause memory overflows. -
Tip: Try to put the
getWherequery to the top of the handler, to make sure it's being preloaded. Read more about how Preload Optimization works.
Modifying Entitiesโ
Use context.Entity.set to create or update an entity:
context.Entity.set({
id: entityId,
...otherEntityFields,
});
Both context.Entity.set and context.Entity.deleteUnsafe methods use the In-Memory Storage under the hood and don't require await in front of them.
Referencing Linked Entitiesโ
When your schema defines a field that links to another entity type, set the relationship using _id with the referenced entity's id. You are storing the ID, not the full entity object.
type A {
id: ID!
b: B!
}
type B {
id: ID!
}
context.A.set({
id: aId,
b_id: bId, // ID of the linked B entity
});
HyperIndex automatically resolves A.b based on the stored b_id when querying the API.
Deleting Entities (Unsafe)โ
To delete an entity:
context.Entity.deleteUnsafe(entityId);
The deleteUnsafe method is experimental and unsafe. You need to manually handle all entity references after deletion to maintain database consistency.
Updating Specific Entity Fieldsโ
Use the following approach to update specific fields in an existing entity:
const pool = await context.Pool.get(poolId);
if (pool) {
context.Pool.set({
...pool,
totalValueLockedETH: pool.totalValueLockedETH.plus(newDeposit),
});
}
context.logโ
The context object also provides a logger that you can use to log messages to the console. Compared to console.log calls, these logs will be displayed on our Envio Cloud runtime logs page.
Read more in the Logging Guide.
context.isPreloadโ
If you need to skip the preload phase for CPU-intensive operations or to perform certain actions only once per event, you can use context.isPreload.
indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event, context }) => {
// Load existing data efficiently
const [sender, receiver] = await Promise.all([
context.Account.getOrThrow(event.params.from),
context.Account.getOrThrow(event.params.to),
]);
// Skip expensive operations during preload
if (context.isPreload) {
return;
}
// CPU-intensive calculations only happen once
const complexCalculation = performExpensiveOperation(event.params.value); // Placeholder function for demonstration
// Create or update sender account
context.Account.set({
id: event.params.from,
balance: sender.balance - event.params.value,
computedValue: complexCalculation,
});
// Create or update receiver account
context.Account.set({
id: event.params.to,
balance: receiver.balance + event.params.value,
});
},
);
Note: While context.isPreload can be useful for bypassing double execution, it's recommended to use the Effect API for external calls instead, as it provides automatic batching and memoization benefits.
External Callsโ
Envio indexer runs using Node.js runtime. This means that you can use fetch or any other library like viem to perform external calls from your handlers.
Note that with Preload Optimization all handlers run twice. But with Effect API this behavior makes your external calls run in parallel, while keeping the processing data consistent.
Check out our IPFS Integration, Accessing Contract State and Effect API guides for more information.
context.effectโ
Define an effect and use it in your handler with context.effect:
// Define an effect that will be called from the handler.
const getMetadata = createEffect(
{
name: "getMetadata",
input: S.string,
output: {
description: S.string,
value: S.bigint,
},
rateLimit: {
calls: 5,
per: "second",
},
cache: true, // Optionally persist the results in the database
},
({ input }) => {
const response = await fetch(`https://api.example.com/metadata/${input}`);
const data = await response.json();
return {
description: data.description,
value: data.value,
};
}
);
indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event, context }) => {
// Load metadata for the token.
// This will be executed in parallel for all events in the batch.
// The call is automatically memoized, so you don't need to worry about duplicate requests.
const sender = await context.effect(getMetadata, event.params.from);
// Process the transfer with the pre-loaded data
},
);
Performance Considerationsโ
For performance optimization and best practices, refer to:
- Benchmarking
- Preload Optimization
These guides offer detailed recommendations on optimizing entity loading and indexing performance.
Block Handlersโ
File: Guides/block-handlers.md
Run logic on every block or an interval.
Understanding Multichain Indexingโ
File: Advanced/multichain-indexing.mdx
For a conceptual overview of what multichain indexing is and when to use it, see What is multichain indexing?. This page covers the HyperIndex configuration and patterns.
Multichain indexing allows you to monitor and process events from contracts deployed across multiple blockchain networks within a single indexer instance. This capability is essential for applications that:
- Track the same contract deployed across multiple networks
- Need to aggregate data from different chains into a unified view
- Monitor cross-chain interactions or state
How It Worksโ
With multichain indexing, events from contracts deployed on multiple chains can be used to create and update entities defined in your schema file. Your blockchain indexer will process events from all configured networks, maintaining proper synchronization across chains.
Configuration Requirementsโ
To implement multichain indexing, you need to:
- Populate the
chainssection in yourconfig.yamlfile for each chain - Specify contracts to index from each chain
- Create event handlers for the specified contracts
Real-World Example: Uniswap V4 Multichain Indexerโ
For a comprehensive, production-ready example of multichain indexing, we recommend exploring our Uniswap V4 Multichain Indexer. This official reference implementation:
- Indexes Uniswap V4 deployments across 10 different blockchain chains
- Powers the official v4.xyz interface with real-time data
- Demonstrates best practices for high-performance multichain indexing
- Provides a complete, production-grade implementation you can study and adapt
!V4 indexer
The Uniswap V4 indexer showcases how to effectively structure a multichain indexer for a complex DeFi protocol, handling high volumes of data across multiple networks while maintaining performance and reliability.
Config File Structure for Multichain Indexingโ
The config.yaml file for multichain indexing contains three key sections:
- Global contract definitions - Define contracts, ABIs, and events once
- Chain-specific configurations - Specify chain IDs and starting blocks
- Contract instances - Reference global contracts with chain-specific addresses
# Example structure (simplified)
contracts:
- name: ExampleContract
abi_file_path: ./abis/example-abi.json
events:
- event: ExampleEvent
chains:
- id: 1 # Ethereum Mainnet
start_block: 0
contracts:
- name: ExampleContract
address: "0x1234..."
- id: 137 # Polygon
start_block: 0
contracts:
- name: ExampleContract
address: "0x5678..."
Key Configuration Conceptsโ
- The global
contractssection defines the contract interface, ABI, handlers, and events once - The
chainssection lists each blockchain chain you want to index - Each chain entry references the global contract and provides the chain-specific address
- This structure allows you to reuse the same handler functions and event definitions across chains
๐ข Best Practice: When developing multichain indexers, append the chain ID to entity IDs to avoid collisions. For example:
user-1for Ethereum anduser-137for Polygon.
Multichain Event Orderingโ
In V3 the indexer always processes multichain events in unordered mode. Events from different chains are processed as soon as they're available, without waiting for the other chains, which keeps latency low.
- Events are still processed in order within each individual chain.
- Events across different chains may be processed out of order.
- Processing happens as soon as events are emitted, so you don't wait for the slowest chain's block time.
This is ideal when:
- Operations on your entities are commutative (order doesn't matter).
- Entities from different chains never interact with each other.
- Processing speed matters more than guaranteed cross-chain ordering.
The V2 unordered_multichain_mode option, the multichain: ordered opt-in, and the UNORDERED_MULTICHAIN_MODE / UNSTABLE__TEMP_UNORDERED_HEAD_MODE environment variables have all been removed in V3 โ there is nothing to configure.
Ordered Multichain Modeโ
HyperIndex V3 doesn't offer an ordered multichain mode, and it's a deliberate choice rather than a feature gap. Ordered mode worked by pausing every chain until events from the slowest chain had caught up, so the moment one chain hiccuped (RPC rate limits, a slow block, a brief reorg) every other chain stalled with it. In practice that meant a multichain indexer was only ever as reliable and as fast as its worst-performing chain, and even healthy chains paid for that coupling with significantly higher latency at the head.
You almost always get better reliability and lower latency by keeping every chain unordered and modeling the cross-chain relationship in your schema instead. Each chain writes a small temporary entity when its side of a cross-chain interaction happens, and the second chain (whichever arrives last) reads those temporary entities and finalizes the unified entity. Because each chain progresses independently, none of them are blocked on each other.
A typical pattern for an A โ B cross-chain message:
type CrossChainMessage {
id: ID! # The shared cross-chain message id (e.g. nonce + originChainId)
sourceChainId: Int
sourceTxHash: String
destinationChainId: Int
destinationTxHash: String
status: String! # "sent" | "delivered"
}
// Chain A: the message was emitted. Create or update the entity with the
// "sent" side of the data. The destination handler may run before or after.
indexer.onEvent(
{ contract: "Bridge", event: "MessageSent" },
async ({ event, context }) => {
const id = `${event.params.originChainId}-${event.params.nonce}`;
const existing = await context.CrossChainMessage.get(id);
context.CrossChainMessage.set({
id,
sourceChainId: event.chainId,
sourceTxHash: event.transaction.hash,
destinationChainId: existing?.destinationChainId,
destinationTxHash: existing?.destinationTxHash,
status: existing?.destinationTxHash ? "delivered" : "sent",
});
},
);
// Chain B: the message was delivered. Read the (maybe-already-existing)
// entity and fill in the destination side, regardless of which side arrived
// first.
indexer.onEvent(
{ contract: "Bridge", event: "MessageDelivered" },
async ({ event, context }) => {
const id = `${event.params.originChainId}-${event.params.nonce}`;
const existing = await context.CrossChainMessage.get(id);
context.CrossChainMessage.set({
id,
sourceChainId: existing?.sourceChainId,
sourceTxHash: existing?.sourceTxHash,
destinationChainId: event.chainId,
destinationTxHash: event.transaction.hash,
status: existing?.sourceTxHash ? "delivered" : "sent",
});
},
);
The same shape works for any "rendezvous" where two chains contribute parts of one logical record (bridges, cross-chain governance, multichain user profiles, etc.). The "temporary" entity is just a regular entity that gets progressively completed as each chain's events arrive โ there's no special API to learn, and the indexer stays fast and resilient because no chain ever waits on another.
If you have a use case that genuinely cannot be expressed this way, reach out on Discord โ we'd like to hear it.
Best Practices for Multichain Indexingโ
1. Entity ID Namespacingโ
Always namespace your entity IDs with the chain ID to prevent collisions between chains. This ensures that entities from different chains remain distinct.
2. Error Handlingโ
Implement robust error handling for chain-specific issues. A failure on one chain shouldn't prevent indexing from continuing on other chains.
3. Testingโ
- Test your indexer with realistic scenarios across all chains
- Use testnet deployments for initial validation
- Verify entity updates work correctly across chains
4. Performance Considerationsโ
- Consider your indexing frequency based on the block times of each chain.
- Monitor resource usage, as indexing multiple chains increases load.
- Adding more chains does not linearly degrade performance โ chains are indexed in parallel.
5. Adding a New Chain to an Existing Indexerโ
To add a new chain to a running indexer:
- Add the new chain entry to your
config.yamlwith the appropriatestart_blockand contract addresses - Push the updated code to your deployment branch (for Envio Cloud) or restart locally with
pnpm envio dev -r
On Envio Cloud, this creates a new deployment that re-indexes all chains (including the new one). Your previous deployment continues serving queries with zero downtime until the new deployment is fully synced. See the deployment guide for details.
Locally, adding a new chain requires a restart and will re-index all chains from their respective start blocks. Note that in V3, envio dev no longer auto-resets the database โ pass -r (or --restart) explicitly when you want a fresh sync. envio start is now production-only.
Troubleshooting Common Issuesโ
-
Entity Conflicts: If you see unexpected entity updates, verify that your entity IDs are properly namespaced with chain IDs.
-
Memory Usage: If your indexer uses excessive memory, consider optimizing your entity structure and implementing pagination in your queries.
Next Stepsโ
- Explore our Uniswap V4 Multichain Indexer for a complete implementation
- Review performance optimization techniques for your indexer
Testingโ
File: Guides/testing.mdx
Introductionโ
Envio ships with a built-in testing library that doubles as a development loop. createTestIndexer() runs your real handlers in-process, so you can iterate on logic and validate behavior without deploying anywhere. It's designed for:
- TDD: Write a failing test, implement the handler, capture the snapshot, commit
- Unit tests: Feed synthetic events directly into handlers to exercise edge cases in isolation
- E2E tests against real blockchain data: Pin a block range or let the indexer auto-detect the first block with events, and run your full handler pipeline end-to-end
- Regression-proof assertions: Inspect entities and per-block change sets, then lock in expected output with
toMatchInlineSnapshot
The library integrates well with Vitest (recommended) and any other JavaScript-based testing framework.
Getting Startedโ
The simplest way to start is auto-exit mode โ no block ranges, no mock events. The indexer automatically finds the first block with events and processes it.
describe("Indexer Testing", () => {
it("Should process first two blocks with events", async (t) => {
const indexer = createTestIndexer();
t.expect(
await indexer.process({ chains: { 1: {} } }),
"Should find the first block with an event on chain 1 and process it."
).toMatchInlineSnapshot(``);
t.expect(
await indexer.process({ chains: { 1: {} } }),
"Should find the second block with an event on chain 1 and process it."
).toMatchInlineSnapshot(``);
});
});
Run pnpm test โ Vitest auto-fills the snapshots on first run. Review and commit them.
Entity State APIโ
Preset state before processing and read entities after.
// Preset state before processing
indexer.EntityName.set({ id: "...", field: value });
// Read state after processing
await indexer.EntityName.get("id"); // returns entity | undefined
await indexer.EntityName.getOrThrow("id"); // throws if not found
await indexer.EntityName.getAll(); // returns all entities of this type
TDD Workflowโ
- Write a failing test with expected entity output
- Implement the handler until the test passes
- Capture the snapshot โ run
pnpm testto filltoMatchInlineSnapshot - Review and commit the snapshot for regression testing
Do not add tests which simply restate the implementation. These provide zero confidence.
Running Testsโ
pnpm test # Run all tests
pnpm test -- -u # Update snapshots
Navigating Hasuraโ
File: Guides/navigating-hasura.md
This page is only relevant when testing on a local machine or using a self-hosted version of Envio that uses Hasura.
Introductionโ
Hasura is a GraphQL engine that provides a web interface for interacting with your indexed blockchain data. When running HyperIndex locally, Hasura serves as your primary tool for:
- Querying indexed data via GraphQL
- Visualizing database tables and relationships
- Testing API endpoints before integration with your frontend
- Monitoring the indexing process
This guide explains how to navigate the Hasura dashboard to effectively work with your indexed data.
Accessing Hasura Consoleโ
When running HyperIndex locally, Hasura Console is automatically available at:
http://localhost:8080
You can access this URL in any web browser to open the Hasura console.
When prompted for authentication, use the password: testing
Key Dashboard Areasโ
The Hasura dashboard has several tabs, but we'll focus on the two most important ones for HyperIndex developers:
API Tabโ
The API tab lets you execute GraphQL queries and mutations on indexed data. It serves as a GraphQL playground for testing your API calls.
Featuresโ
- Explorer Panel: The left panel shows all available entities defined in your
schema.graphqlfile - Query Builder: The center area is where you write and execute GraphQL queries
- Results Panel: The right panel displays query results in JSON format
Available Entitiesโ
By default, you'll see:
- All entities defined in your
schema.graphqlfile dynamic_contracts(for dynamically added contracts)raw_eventstable (Note: This table is no longer populated by default to improve performance. To enable storage of raw events, addraw_events: trueto yourconfig.yamlfile as described in the Raw Events Storage section)
Example Queryโ
Try a simple query to test your blockchain indexer:
query MyQuery {
User(limit: 5) {
id
latestGreeting
numberOfGreetings
}
}
Click the "Play" button to execute the query and see the results.
For more advanced GraphQL query options, see Hasura's quickstart guide.
Data Tabโ
The Data tab provides direct access to your database tables and relationships, allowing you to view the actual indexed data.
Featuresโ
- Schema Browser: View all tables in the database (left panel)
- Table Data: Examine and browse data within each table
- Relationship Viewer: See how different entities are connected
Working with Tablesโ
- Select any table from the "public" schema to view its contents
- Use the "Browse Rows" tab to see all data in that table
- Check the "Insert Row" tab to manually add data (useful for testing)
- View the "Modify" tab to see the table structure
Verifying Indexed Dataโ
To confirm your blockchain indexer is working correctly:
- Check entity tables to ensure they contain the expected data
- Query the
envio_chainstable (or use the Metadata Query API) to see each chain's latest processed block and confirm the indexer is making progress
Common Tasksโ
Checking Indexing Statusโ
To verify your blockchain indexer is actively processing new blocks:
- Go to the Data tab
- Select the
envio_chainstable (or query the Metadata Query API) to see each chain's latest processed block - Monitor those values over time to ensure they're advancing
(Note the TUI is also an easy way to monitor this)
Troubleshooting Missing Dataโ
If expected data isn't appearing:
- Check if you've enabled raw events storage (
raw_events: trueinconfig.yaml) and then examine theraw_eventstable to confirm events were captured - Verify your event handlers are correctly processing these events
- Examine your GraphQL queries to ensure they match your schema structure
- Check console logs for any processing errors
Resetting Indexed Dataโ
When testing, you may need to reset your database:
- Stop your indexer
- Reset your database (refer to the development guide for commands)
- Restart your indexer to begin processing from the configured start block
Best Practicesโ
- Regular Verification: Periodically check both the API and Data tabs to ensure your blockchain indexer is functioning correctly
- Query Testing: Test complex queries in the API tab before implementing them in your application
- Schema Validation: Use the Data tab to verify that relationships between entities are correctly established
- Performance Monitoring: Watch for tables that grow unusually large, which might indicate inefficient indexing
Aggregations: local vs hosted (avoid the footโgun)โ
When developing locally with Hasura, you may notice that GraphQL aggregate helpers (for example, count/sum-style aggregations) are available. On Envio Cloud, these aggregate endpoints are intentionally not exposed. Aggregations over large datasets can be very slow and unpredictable in production.
The recommended approach is to compute and store aggregates at indexing time, not at query time. In practice this means maintaining counters, sums, and other rollups in entities as part of your event handlers, and then querying those precomputed values.
Example: indexing-time aggregationโ
schema.graphql
# singleton; you hardcode the id and load it in and out
type GlobalState {
id: ID! # "global-state"
count: Int!
}
type Token {
id: ID! # incremental number
description: String!
}
EventHandler.ts
const globalStateId = "global-state";
indexer.onEvent(
{ contract: "NftContract", event: "Mint" },
async ({ event, context }) => {
const globalState = await context.GlobalState.get(globalStateId);
if (!globalState) {
context.log.error("global state doesn't exist");
return;
}
const incrementedTokenId = globalState.count + 1;
context.Token.set({
id: incrementedTokenId,
description: event.params.description,
});
context.GlobalState.set({
...globalState,
count: incrementedTokenId,
});
},
);
This pattern scales: you can keep per-entity counters, rolling windows (daily/hourly entities keyed by date), and top-N caches by updating entities as events arrive. Your queries then read these precomputed values directly, avoiding expensive runtime aggregations.
Exceptional casesโ
If runtime aggregate queries are a hard requirement for your use case, please reach out and we can evaluate options for your project on Envio Cloud. Contact us on Discord.
Disable Hasura for Self-Hosted Blockchain Indexersโ
Set the ENVIO_HASURA environment variable to false to disable Hasura integration for self-hosted blockchain indexers.
Environment Variablesโ
File: Guides/environment-variables.md
Environment variables are a crucial part of configuring your Envio blockchain indexer. They allow you to manage sensitive information and configuration settings without hardcoding them in your codebase.
Naming Conventionโ
All environment variables used by Envio must be prefixed with ENVIO_. This naming convention:
- Prevents conflicts with other environment variables
- Makes it clear which variables are used by the Envio indexer
- Ensures consistency across different environments
Envio API Token (required for HyperSync)โ
To ensure continued access to HyperSync, set an Envio API token in your environment.
- Use
ENVIO_API_TOKENto provide your token at runtime - See the API Tokens guide for how to generate a token: API Tokens
- A token is only required when using Envio as the data provider (HyperSync). Indexers that source data from an external RPC don't need one.
Envio-specific environment variablesโ
The following variables are used by HyperIndex:
-
ENVIO_API_TOKEN: API token for HyperSync access (required when indexing via HyperSync โ get one at envio.dev/app/api-tokens) -
ENVIO_HASURA: Set tofalseto disable Hasura integration for self-hosted blockchain indexers -
ENVIO_TUI: Set tofalseto disable the terminal UI (replaces the V2TUI_OFF=trueflag; the TUI is also auto-disabled in CI and under AI agents) -
ENVIO_PG_PORT: Port for the Postgres service used by HyperIndex during local development -
ENVIO_PG_PASSWORD: Postgres password (self-hosted) -
ENVIO_PG_USER: Postgres username (self-hosted) -
ENVIO_PG_DATABASE: Postgres database name (self-hosted) -
ENVIO_PG_SCHEMA: Postgres schema name override for the generated/public schema (replacesENVIO_PG_PUBLIC_SCHEMA; the old name is still accepted until v4)
The V2 variables MAX_BATCH_SIZE, ENVIO_INDEXING_BLOCK_LAG, UNORDERED_MULTICHAIN_MODE, and UNSTABLE__TEMP_UNORDERED_HEAD_MODE have been removed in V3. Use the full_batch_size config option in config.yaml instead of MAX_BATCH_SIZE, and use the per-chain block_lag option instead of ENVIO_INDEXING_BLOCK_LAG. Unordered multichain processing is now the default.
Example Environment Variablesโ
Here are some commonly used environment variables:
# Envio API Token (required for HyperSync access)
ENVIO_API_TOKEN=your-secret-token
# Blockchain RPC URL
ENVIO_RPC_URL=https://arbitrum.direct.dev/your-api-key
# Coingecko API key
ENVIO_COINGECKO_API_KEY=api-key
# Disable the terminal UI
ENVIO_TUI=false
Setting Environment Variablesโ
Local Developmentโ
For local development, you can set environment variables in several ways:
- Using a
.envfile in your project root:
# .env
ENVIO_API_TOKEN=your-secret-token
ENVIO_RPC_URL=https://arbitrum.direct.dev/your-api-key
- Directly in your terminal:
export ENVIO_API_TOKEN=your-secret-token
export ENVIO_RPC_URL=https://arbitrum.direct.dev/your-api-key
Envio Cloudโ
When using Envio Cloud, you can configure environment variables through the Envio platform's dashboard. Remember that all variables must still be prefixed with ENVIO_.
For more information about environment variables in Envio Cloud, see the Envio Cloud documentation.
Configuration Fileโ
For use of environment variables in your configuration file, read the docs here: Configuration File.
Best Practicesโ
- Never commit sensitive values: Always use environment variables for sensitive information like API keys and database credentials
- Never commit or use private keys: Never commit or use private keys in your codebase
- Use descriptive names: Make your environment variable names clear and descriptive
- Document your variables: Keep a list of required environment variables in your project's README
- Use different values: Use different environment variables for development, staging, and production environments
- Validate required variables: Check that all required environment variables are set before starting your blockchain indexer
Troubleshootingโ
If you encounter issues with environment variables:
- Verify that all required variables are set
- Check that variables are prefixed with
ENVIO_ - Ensure there are no typos in variable names
- Confirm that the values are correctly formatted
For more help, see our Troubleshooting Guide.
Uniswap V4 Multichain Indexerโ
File: Examples/example-uniswap-v4.md
The following blockchain indexer example is a reference implementation and can serve as a starting point for applications with similar logic.
This official Uniswap V4 indexer is a comprehensive implementation for the Uniswap V4 protocol using Envio HyperIndex. This is the same indexer that powers the v4.xyz website, providing real-time data for the Uniswap V4 interface.
Key Featuresโ
- Multichain Support: Indexes Uniswap V4 deployments across 15 different blockchain networks in real-time
- Complete Pool Metrics: Tracks pool statistics including volume, TVL, fees, and other critical metrics
- Swap Analysis: Monitors swap events and liquidity changes with high precision
- Hook Integration: In-progress support for Uniswap V4 hooks and their events
- Production Ready: Powers the official v4.xyz interface with production-grade reliability
- Ultra-Fast Syncing: Processes massive amounts of blockchain data significantly faster than alternative blockchain indexing solutions, reducing sync times from days to minutes
!V4 gif
Technical Overviewโ
This indexer is built using TypeScript and provides a unified GraphQL API for accessing Uniswap V4 data across all supported networks. The architecture is designed to handle high throughput and maintain consistency across different blockchain networks.
Performance Advantagesโ
The Envio-powered Uniswap V4 indexer offers extraordinary performance benefits:
- 10-100x Faster Sync Times: Leveraging Envio's HyperSync technology, this indexer can process historical blockchain data orders of magnitude faster than traditional solutions
- Real-time Updates: Maintains low latency for new blocks while efficiently managing historical data
Use Casesโ
- Power analytics dashboards and trading interfaces
- Monitor DeFi positions and protocol health
- Track historical performance of Uniswap V4 pools
- Build custom notifications and alerts
- Analyze hook interactions and their impact
Getting Startedโ
To use this indexer, you can:
- Clone the repository
- Follow the installation instructions in the README
- Run the indexer locally or deploy it to a production environment
- Access indexed data through the GraphQL API
Contributionโ
The Uniswap V4 indexer is actively maintained and welcomes contributions from the community. If you'd like to contribute or report issues, please visit the GitHub repository.
This is an official reference implementation that powers the v4.xyz website. While extensively tested in production, remember to validate the data for your specific use case. The indexer is continuously updated to support the latest Uniswap V4 features and optimizations.
Sablier Protocol Indexersโ
File: Examples/example-sablier.md
The following blockchain indexers serve as exceptional reference implementations for the Sablier protocol, showcasing professional development practices and efficient multichain data processing.
Overviewโ
Sablier is a token streaming protocol that enables real-time finance on the blockchain, allowing tokens to be streamed continuously over time. These official Sablier indexers track streaming activity across many EVM-compatible chains, providing comprehensive data through a unified GraphQL API.
Professional Indexer Suiteโ
Sablier maintains two public indexers, each targeting a specific part of their protocol:
1. Streams Indexerโ
Tracks Sablier's payment-stream data across both the Lockup and Flow products. Lockup covers streams with fixed durations and amounts (creation, cancellation, and withdrawal events), while Flow covers open-ended streaming with dynamic flow rates. For more detail, see the Lockup indexer docs and the Flow indexer docs.
2. Airdrops Indexerโ
Tracks Sablier's Merkle airdrop campaigns, which enable efficient batch stream creation using cryptographic proofs. This indexer captures data about campaign creation, claims, and related activity, powering both Airstreams and Instant Airdrops. For more detail, see the Airdrops indexer docs.
Key Featuresโ
- Comprehensive Multichain Support: Indexes data across many EVM-compatible chains
- Professionally Maintained: Used in production by the Sablier team and their partners
- Extensive Test Coverage: Includes comprehensive testing to ensure data accuracy
- Optimized Performance: Implements efficient data processing techniques
- Well-Documented: Clear code structure with extensive comments
- Backward Compatibility: Carefully manages schema evolution and contract upgrades
- Cross-chain Architecture: Envio promotes efficient cross-chain indexing where all networks share the same indexer endpoint
Best Practices Showcaseโ
These blockchain indexers demonstrate several development best practices:
- Modular Code Structure: Well-organized code with clear separation of concerns
- Consistent Naming Conventions: Professional and consistent naming throughout
- Efficient Event Handling: Optimized processing of blockchain events
- Comprehensive Entity Relationships: Well-designed data model with proper relationships
- Thorough Input Validation: Robust error handling and input validation
- Detailed Changelogs: Documentation of breaking changes and migrations
- Preload Optimization: Envio indexers benefit from always-on Preload Optimization, which batches entity reads and runs external calls in parallel through the Effect API
Getting Startedโ
To use these indexers as a reference for your own development:
- Clone the indexer that matches your needs:
- Review the file structure and implementation patterns
- Examine the event handlers for efficient data processing techniques
- Study the schema design for effective entity modeling
For complete API documentation and usage examples, see:
These are official indexers maintained by the Sablier team and represent production-quality implementations. They serve as an excellent example of professional blockchain indexer development and are regularly updated to support the latest protocol features.
Hosted Serviceโ
File: Hosted_Service/hosted-service.md
Envio Cloud (formerly Hosted Service) is a fully managed hosting solution for your blockchain indexers, providing all the infrastructure, scaling, and monitoring needed to run production-grade indexers without operational overhead.
Envio Cloud offers multiple plans to suit different needs, from free development environments to enterprise-grade dedicated hosting. Each plan includes powerful features like static production endpoints, built-in alerts, and production-ready infrastructure.
Deployment Optionsโ
Envio provides flexibility in how you deploy and host your indexers:
-
Envio Cloud (Fully Managed): Let Envio handle everything. The following sections of this page outline Envio Cloud in more detail. This is the recommended deployment method for most users and removes the hosting overhead for your team. See below for the all the awesome features we provide and see the Pricing & Billing page for more information on which plan suits your indexing needs.
-
Self-Hosting: Run your indexer on your own infrastructure. This requires advanced setup and infrastructure knowledge not unique to Envio. See the following repository for a simple docker example to get you started. Please note this example does not cover all infrastructure related needs. It is recommended that at least a separate Postgres management tool is used for self-hosting in production. For further instructions see the Self Hosting Guide
Key Featuresโ
- Git-based Deployments: Similar to Vercel, deploy your indexer by simply pushing to a designated deployment branch
- Zero Infrastructure Management: We handle all the servers, databases, and scaling for you
- Static Production Endpoints: Consistent URLs with zero-downtime deployments and instant version switching
- Built-in Monitoring: Track logs, sync status, and deployment health in real-time
- Comprehensive Alerting: Multi-channel notifications (Discord, Slack, Telegram, Email) for critical issues, performance warnings, and deployment updates
- Security Features: IP/Domain whitelisting to control access to your indexer endpoints
- GraphQL API: Access your indexed data through a performant, production-ready GraphQL endpoint
- Multichain Support: Deploy indexers that track multiple networks from a single codebase
Deployment Modelโ
Envio Cloud provides a seamless GitHub-integrated deployment workflow:
- GitHub Integration: Install the Envio Deployments GitHub App to connect your repositories
- Flexible Configuration: Support for monorepos with configurable root directories, config file locations, and deployment branches
- Automatic Deployments: Push to your deployment branch to trigger builds and deployments
- Version Management: Maintain multiple deployment versions with one-click switching and rollback capabilities
- Real-time Monitoring: Track deployment progress, logs, and sync status through the dashboard
Multiple Indexers: Deploy several indexers from a single repository using different configurations, branches, or directories.
You can view and manage your hosted indexers in the Envio Explorer.
Getting Startedโ
- Features - Learn about all available Envio Cloud features
- Deployment Guide - Step-by-step instructions for deploying your indexer
- Envio Cloud CLI - Manage and monitor your hosted indexers from the command line
- Pricing & Billing - Compare plans and pricing options
- Self-Hosting - Run your indexer on your own infrastructure
It is recommended that before deploying to Envio Cloud, the indexer is built and tested locally to ensure it runs smoothly. For a complete list of local CLI commands to develop your indexer, see the CLI Commands documentation.
Envio Cloud Featuresโ
File: Hosted_Service/hosted-service-features.md
Envio Cloud includes several production-ready features to help you manage and secure your blockchain indexer deployments.
Most features listed on this page are available for paid production plans only. The free development plan has limited features and is designed for testing and development purposes. View our pricing plans to see what's included in each plan.
Deployment Tagsโ
Organize and identify your deployments with custom key/value tags. Tags help you categorize deployments by environment, project, team, or any custom attribute that fits your workflow.
How it works:
- Add up to 5 custom tags per deployment via the deployment overview page
- Each tag consists of a key (max 20 characters) and a value (max 20 characters, automatically lowercased)
- Click "+ Add Tag" to create new tags, or click existing tags to edit or delete them
Special name Tag:
The name tag has special behaviorโwhen set, its value is displayed directly on the deployment list, making it easy to identify deployments at a glance without navigating into each one.
Example Use Cases:
name: stagingorname: productionโ quickly identify deployment purposeenv: staging/env: productionโ categorize by environmentteam: frontendโ organize by team ownershipversion: v2โ track deployment versions
Benefits:
- Quickly identify deployments in the list view
- Organize deployments across multiple projects or environments
- Add context and metadata to your deployments
- Filter and locate deployments more efficiently
IP Whitelistingโ
Availability: Paid plans only
Control access to your indexer by restricting requests to specific IP addresses. This security feature helps protect your data and ensures only authorized clients can query your indexer.
Benefits:
- Enhanced security for sensitive data
- Prevent unauthorized access
- Control API usage from specific sources
- Ideal for production environments with strict access requirements
Effect API Cacheโ
Availability: Medium plans and up
Speed up your indexer deployments by caching Effect API results. When enabled, new deployments will start with preloaded effect data, eliminating the need to re-fetch external data and significantly reducing sync time.
How it works:
- Save a Cache: From any deployment, click "Save Cache" to capture the current effect data
- Configure Settings: Navigate to Settings > Cache to manage your caches
- Enable Caching: Toggle caching on and select which cache to use for new deployments
- Deploy: New deployments will automatically restore from the selected cache
Key Features:
- Quick Save: Save cache directly from the deployment page with one click
- Cache Management: View, select, and delete caches from the Cache settings page
- Automatic Restore: New deployments preload effect data from the active cache
- Download Cache: Download caches for local development, enabling faster iteration without re-fetching external data
Benefits:
- Dramatically faster deployment sync times
- Reduced external API calls during indexing
- Seamless deployment updates with preserved effect state
Learn more about the Effect API and how caching works in our Effect API documentation.
This feature is only available for blockchain indexers deployed with version 2.26.0 or higher.
Built-in Alertsโ
Availability: Paid plans only
Stay informed about your indexer's health and performance with our integrated alerting system. Configure multiple notification channels and choose which alerts you want to receive.
This feature is only available for blockchain indexers deployed with version 2.24.0 or higher.
Notification Channelsโ
Configure one or multiple notification channels to receive alerts:
- Discord
- Slack
- Telegram
Zero-Downtime Deploymentsโ
Update your blockchain indexer without any service interruption using our seamless deployment system with static production endpoints.
How it works:
- Deploy new versions alongside your current deployment
- Each indexer gets a static production endpoint that remains consistent
- Use 'Promote to Production' to instantly route the static endpoint to any deployment
- All requests to your static production endpoint are automatically routed to the promoted deployment
- Maintain API availability throughout upgrades with no endpoint changes required
Key Features:
- Static Production Endpoint: Consistent URL that never changes, regardless of which deployment is active
- Instant Switching: Promote any deployment to production with zero downtime
- Rollback Capabilities: Quickly switch back to previous deployments if needed
- Seamless Updates: Your applications continue working without any configuration changes
Deployment Location Choiceโ
Full support for cross-region deployments is in active development. If you require a deployment to be based in the USA please contact us through our support channel on discord.
Availability: Dedicated plans only
Choose your primary deployment region to optimize performance and meet compliance requirements.
Available Regions:
- USA
- EU
Benefits:
- Reduced latency for your target users
- Data residency compliance support
- Custom infrastructure configurations
- Dedicated infrastructure resources
Direct Database Accessโ
Availability: Dedicated plans only
Access your indexed data directly through SQL queries, providing flexibility beyond the standard GraphQL endpoint.
Use Cases:
- Complex analytical queries
- Custom data exports
- Advanced reporting and dashboards
- Integration with external analytics tools
Powerful Analytics Solutionโ
Availability: Dedicated plans only (additional cost)
A comprehensive analytics platform that automatically pipes your indexed data from PostgreSQL into ClickHouse (approximately 2 minutes behind real-time) and provides access through a hosted Metabase instance.
Technical Architecture:
- Data Pipeline: Automatic replication from PostgreSQL to ClickHouse
- Near Real-time: Data available in an analytics platform within ~2 minutes
- Frontend: Hosted Metabase instance for visualization and analysis
- Performance: ClickHouse optimized for analytical queries on large datasets
Capabilities:
- Interactive, customizable dashboards through Metabase
- Variety of visualization options (charts, graphs, tables, maps)
- Fast analytical queries on large datasets via ClickHouse
- Ad-hoc SQL queries for data exploration
- Automated alerts based on data thresholds
- Team collaboration and report sharing
- Export capabilities for further analysis
For deployment instructions and limits, see our Deployment Guide. For pricing and feature availability by plan, see our Billing & Pricing page.
Deploying Your Indexerโ
File: Hosted_Service/hosted-service-deployment.md
Envio Cloud provides a seamless git-based deployment workflow, similar to modern platforms like Vercel. This enables you to easily deploy, update, and manage your blockchain indexers through your normal development workflow.
Prerequisites & Important Informationโ
Requirementsโ
-
Version Support: We strongly advise using the latest release version for improved deployment performance. Envio Cloud requires a minimum version of at least
2.21.5. Additionally, the following versions are not supported on Envio Cloud:2.29.x
-
PNPM Support: the deployment must be compatible with pnpm version
10.32.0 -
Repository Folder:
- Package.json: a
package.jsonfile must be present in the root folder and support the above two requirements, with the envio version explicitly configured in the dependencies. - Configuration file: a HyperIndex configuration file must be present.
The root folder and configuration file name can be set in the indexer settings.
- Package.json: a
-
GitHub Repository: The repository must be no larger than
100MB. Caching between deployments is supported for paid plans using the Effects Api. -
Node Version: It is strongly recommended that the indexer is compatible with node version 24 or higher.
Before deploying your indexer, please be aware of the below limits and policies
Deployment Limitsโ
- 3 development plan indexers per organization
- Deployments per indexer: 3 deployments per indexer
- Deployments can be deleted in Envio Cloud to make space for more deployments
Development Plan Fair Usage Policyโ
The free development plan includes automatic deletion policies to ensure fair resource allocation:
Automatic Deletion Rules:โ
- Hard Limits:
- Deployments that exceed 20GB of storage will be automatically deleted
- Deployments older than 30 days will be automatically deleted
- Soft Limits (whichever comes first):
- 100,000 events processed
- 5GB storage used
- no requests for 7 days
When soft limits are breached, the two-stage deletion process begins
Two-Stage Deletion Processโ
Applies to development deployments that breach the soft limits
- Grace Period (7 days) - Your indexer continues to function normally, you receive notification about the upcoming deletion
- Read-Only Access (3 days) - Indexer stops processing new data, existing data remains accessible for queries
- Full Deletion - Indexer and all data are permanently deleted
The grace period durations (7 + 3 days) are subject to change. Always monitor your deployment status and upgrade when approaching limits.
For complete pricing details and feature comparison, see our Pricing & Billing page.
Step-by-Step Deployment Instructionsโ
Initial Setupโ
- Log in with GitHub: Visit the Envio App and authenticate with your GitHub account
- Select an Organization: Choose your personal account or any organization you have access to
- Install the Envio Deployments GitHub App: Grant access to the repositories you want to deploy
Configure Your Indexerโ
- Connect a Repo: Select the repository containing your indexer code
- Add the Indexer: Click "Add Indexer" and configure your indexer
- Configure Deployment Settings:
- Specify the config file location
- Set the root directory (important for monorepos)
- Choose the deployment branch
You can deploy multiple indexers from a single repository by configuring them with different config file paths, root directories, and/or deployment branches.
If you're working in a monorepo, ensure all your imports are contained within your indexer directory to avoid deployment issues.
Deploy Your Codeโ
- Create a Deployment Branch: Set up the branch you specified during configuration
- Deploy via Git: Push your code to the deployment branch
- Monitor Deployment: Track the progress of your deployment in the Envio dashboard
Manage Your Deploymentโ
- Version Management: Once deployed, you can:
- View detailed logs
- Switch between different deployed versions
- Rollback to previous versions if needed
Updating Your Deploymentโ
After your initial deployment, you can update your indexer by pushing new commits to the deployment branch. Each push creates a new deployment version.
What happens on each pushโ
When you push to your deployment branch, Envio Cloud will:
- Build your updated indexer code
- Start a new deployment that re-indexes from the start block
- Keep your previous deployment running and serving queries until the new one is fully synced
This means there is no downtime during updates โ your existing deployment continues serving data while the new one catches up.
When re-indexing is requiredโ
A full re-index from the start block happens on every new deployment. This includes changes to:
- Event handler logic
- Schema (
schema.graphql) - Configuration (
config.yaml) - ABIs or contract addresses
Use the Effects API cache to speed up re-indexing by caching expensive external calls (like eth_call results) across deployments. This is available on paid plans.
Adding a new chain to your indexerโ
To add a new chain, update your config.yaml with the new network configuration and push to the deployment branch. The new deployment will index all configured chains, including the new one.
Your previous deployment continues serving data for the existing chains while the new deployment syncs.
Rolling back to a previous versionโ
If a new deployment introduces issues, you can switch back to a previous version from the Envio Cloud dashboard. Navigate to your indexer and select the version you want to activate.
Monitoringโ
Once your indexer is deployed, you can monitor its health, performance, and progress using several built-in tools including the dashboard, logs, and alerts.
For detailed information about monitoring your deployments, see our Monitoring Guide.
Continuous Deployment Best Practices and Configurationโ
For a robust deployment workflow, we recommend:
- Protected Branches: Set up branch protection rules for your deployment branch
- Pull Request Workflow: Instead of pushing directly to the deployment branch, use pull requests from feature branches
- CI Integration: Add tests to your CI pipeline to validate indexer functionality before merging to the deployment branch
Continuous Configurationโ
After deploying your indexer, you can manage its configuration through the Settings tab in the Envio Cloud dashboard:
General Tabโ
The General tab provides core configuration options:
- Config File Path: Update the location of your indexer's configuration file
- Deployment Branch: Change which Git branch triggers deployments
- Root Directory: Modify the root directory for your indexer (useful for monorepos)
- Delete Indexer: Permanently remove the indexer and all its deployments
Deleting an indexer is permanent and will remove all associated deployments and data. This action cannot be undone.
Environment Variables Tabโ
Configure environment-specific variables for your indexer:
- Add custom environment variables with the
ENVIO_prefix - Environment variables are securely stored and injected into your indexer at runtime
- Useful for API keys, configuration values, and other deployment-specific settings
Use environment variables for sensitive data rather than hardcoding values in your repository. Remember to prefix all variables with ENVIO_.
Plans & Billing Tabโ
Manage your indexer's pricing plan and billing:
- Select from available pricing plans
- Upgrade your plan to suit your needs
- View current plan features and limits
For detailed pricing information, see our Pricing & Billing page.
Alerts Tabโ
Configure monitoring and notification preferences:
- Set up notification channels (Discord, Slack, Telegram, Email)
- Choose which alert types to receive (Production Endpoint Down, Indexer Stopped Processing, etc.)
- Configure deployment notifications (Historical Sync Complete)
For complete alert configuration details, see our Features page.
Alert configuration is available for indexers deployed with version 2.24.0 or higher on paid production plans.
Visual Reference Guideโ
The following screenshots show each step of the deployment process:
Step 1: Select Organizationโ
!Select organisation
Step 2: Install GitHub Appโ
!Install GitHub App
Step 3: Connect a Repoโ
!Connect a repo
Step 4: Add the Indexerโ
!Add the indexer
Step 5: Configure Deployment Settingsโ
!Configure indexer
Step 6: Create a Deployment Branchโ
!Create deployment branch
Step 7: Deploy via Gitโ
!Deploy via Git
Step 8: Indexer Deployedโ
Once deployment completes, your indexer should be live and you should see the overview dashboard below. Full monitoring details are available in our Monitoring Guide.
!Indexer overview
Step 9: Manage Indexer Configurationโ
Manage indexer configurations and deployments using the sidebar navigation on the left.
!Manage indexer configuration
Related Documentationโ
- Features - Learn about all available Envio Cloud features
- Envio Cloud CLI - Deploy and manage indexers from the command line
- Pricing & Billing - Compare plans and see feature availability
- Overview - Introduction to Envio Cloud
- Self-Hosting - Run your indexer on your own infrastructure
Monitoring Your Blockchain Indexerโ
File: Hosted_Service/hosted-service-monitoring.md
Once your blockchain indexer is deployed, Envio Cloud provides several tools to help you monitor its health, performance, and progress.
Dashboard Overviewโ
The main dashboard provides real-time visibility into your indexer's status:
Key Metrics Displayed:
- Active Deployments: Track how many deployments are currently running (e.g., 1/3 active)
- Deployment Status: See whether your indexer is actively syncing, stopped, or has encountered errors
- Recent Commits: View your deployment history with commit information and active status
- Usage Statistics: Monitor your indexing hours, storage usage, and query rate limits
- Network Progress: Real-time progress bars showing sync status for each blockchain network
- Events Processed: Track the total number of events your indexer has processed
- Historical Sync Time: See how long it took to complete the initial sync
Deployment Status Indicatorsโ
Each deployment shows clear status information:
- Syncing: Indexer is actively processing blocks and events
- Syncing Stopped: Indexer has stopped processing (may indicate an error or a breach of plan limits)
- Historical Sync Complete: Initial sync finished, indexer is processing new blocks in real-time
Error Detection and Troubleshootingโ
When issues occur, the dashboard displays failure information to help you quickly diagnose problems.
Failure Information Includes:
- Error Type: Clear indication of the failure (e.g., "Indexing Has Stopped")
- Error Description: Details about what went wrong (e.g., "Error during event handling")
- Next Steps: Guidance on where to find more information (error logs)
- Support Access: Direct link to Discord for assistance
Loggingโ
Full logging supported is integrated and configured by Envio via Envio Cloud
Access detailed logs to troubleshoot issues and monitor indexer behavior:
- Real-time Logs: View live logs as your indexer processes events
- Error Logs: Quickly identify and diagnose errors in your event handlers
- Deployment Logs: Track the deployment process and startup sequence
- Filter Log Levels: Find specific log entries to debug issues
Access logs through the "Logs" button on your deployment page.
Built-in Alertsโ
Configure proactive monitoring through the Alerts tab to receive notifications before issues impact your users:
- Critical Alerts: Get notified when your production endpoint goes down
- Warning Alerts: Receive alerts when your indexer stops processing blocks
- Info Alerts: Stay informed about indexer restarts and error logs
- Deployment Notifications: Know when historical sync completes
For detailed alert configuration, see the Deployment Guide and our Features page.
Notification Channelsโ
Envio Cloud supports the following notification channels:
| Channel | Configuration |
|---|---|
| Discord | Webhook URL, optional bot username |
| Slack | Webhook URL, channel name, optional bot username |
| Telegram | Bot API token, chat ID |
| One or more email addresses | |
| Webhook | Any HTTP endpoint URL (works with incident.io, PagerDuty, Opsgenie, etc.) |
Channels are configured at the organisation level via Settings > Notification Channels, then subscribed to alerts on individual indexers.
Webhook Channelโ
The webhook channel sends a structured JSON payload via HTTP POST to any URL you provide. This makes it compatible with any service that accepts webhook-based alerts.
Custom Headers (optional): When creating a webhook channel, you can add custom HTTP headers that will be sent with every request. This is useful for services that require authentication via API keys or bearer tokens โ for example, incident.io requires an Authorization: Bearer header.
The webhook channel is a generic HTTP endpoint. It is not guaranteed to work with all third-party services โ please see the integration guides below for supported setup instructions. If you need help integrating with a specific service, please reach out on the Envio Discord.
Webhook Payload Schema:
{
"title": "IndexerStoppedProcessing โ my-indexer (abc123)",
"status": "firing",
"severity": "warning",
"description": "Indexer my-indexer has stopped processing blocks for 10+ minutes (commit: abc123)",
"source_url": "https://envio.dev/app/my-org/my-indexer/abc123",
"alert_id": "my-org/proj456/my-indexer/IndexerStoppedProcessing",
"metadata": {
"organisationId": "my-org",
"indexerName": "my-indexer",
"commit": "abc123",
"labels": { "envio_alert_name": "IndexerStoppedProcessing", "severity": "warning" },
"annotations": {
"summary": "Indexer my-indexer has stopped processing blocks for 10+ minutes",
"description": "The indexer has not processed any blocks in the last 10 minutes."
},
"startsAt": "2025-05-15T12:00:00Z",
"type": "alert"
}
}
The endsAt field is only included when the alert has resolved. Firing alerts omit this field.
| Field | Description |
|---|---|
title | Alert name (e.g. IndexerStoppedProcessing, ProdEndpointDown) |
status | "firing" or "resolved" |
severity | "critical", "warning", or "info" |
description | Human-readable summary of the alert |
source_url | Link to the deployment in the Envio dashboard |
alert_id | Unique key for deduplication: orgId/projectId/indexerName/alertName |
metadata | Additional context including labels, timestamps, and deployment info |
Webhook Integrationsโ
incident.ioโ
incident.io can receive Envio alerts via their Custom HTTP Sources feature.
Step 1: Create a Custom HTTP Source in incident.io
In the incident.io dashboard, Follow the incident.io custom HTTP sources guide to setup the webhook integration.
Step 2: Configure the Transform Expression
Paste this ES5 JavaScript transform expression to map the Envio payload into incident.io's format:
var severity = body.severity || "info";
var severityMap = { critical: 1, warning: 2, info: 3 };
return {
title: body.title,
status: body.status,
description: body.description || "",
source_url: body.source_url || "",
metadata: {
severity: severity,
severity_rank: severityMap[severity] || 3,
organisation_id: body.metadata.organisationId,
indexer_name: body.metadata.indexerName,
commit: body.metadata.commit,
source: "envio",
type: body.metadata.type,
starts_at: body.metadata.startsAt,
ends_at: body.metadata.endsAt || "",
labels: JSON.stringify(body.metadata.labels || {}),
annotations: JSON.stringify(body.metadata.annotations || {})
}
};
Step 3: Set the Deduplication Key Path
Set the dedup key path to alert_id. This ensures alerts are grouped per indexer and auto-resolve when the status changes to "resolved".
Step 4: Add the Webhook Channel in Envio
- Go to Settings > Notification Channels in the Envio Cloud dashboard
- Click Add Channel and select Webhook
- Paste the webhook URL from incident.io
- Expand Custom Headers and add the authorization header:
- Header name:
Authorization - Value: Copy the full
Bearervalue from the incident.io source config
- Header name:
- Subscribe your indexer's alerts to this channel
Set up multiple notification channels (Paid Plans Only) to ensure you never miss critical alerts about your indexer's health.
Visual Referenceโ
Dashboard Overviewโ
!Dashboard overview
Network Progress by Chainโ
!Network progress bars
Example Failure Notificationโ
When indexing stops, the dashboard clearly surfaces the issue so you can investigate and resolve it quickly.
!Indexing has stopped
Related Documentationโ
- Deploying Your Indexer - Complete deployment guide
- Envio Cloud CLI - Monitor deployments from the command line with
envio-cloud deployment metricsandenvio-cloud deployment status - Features - Learn about all available Envio Cloud features
- Pricing & Billing - Compare plans and see feature availability
Envio Cloud CLIโ
File: Hosted_Service/envio-cloud-cli.md
The envio-cloud CLI is a command-line tool for interacting with Envio Cloud. It enables you to deploy, manage, and monitor your blockchain indexers directly from the terminal โ making it particularly useful for CI/CD pipelines, scripting, and agentic workflows.
Installationโ
npm install -g envio-cloud
Or run directly without installation:
npx envio-cloud
Shell Completionโ
The envio-cloud CLI ships with shell completion scripts for bash, zsh, fish, and powershell. Completion includes dynamic suggestions for indexer names and commit hashes, so you can tab-complete them directly from the terminal.
Run the one-liner for your shell to install completions:
| Shell | One-liner |
|---|---|
zsh | echo 'source > ~/.zshrc |
bash | envio-cloud completion bash > ~/.local/share/bash-completion/completions/envio-cloud |
fish | envio-cloud completion fish > ~/.config/fish/completions/envio-cloud.fish |
powershell | envio-cloud completion powershell >> $PROFILE |
Restart your shell (or source your profile) for the completions to take effect. Run envio-cloud completion --help for further options.
Authenticationโ
Browser Loginโ
envio-cloud login
Opens browser-based authentication via envio.dev with a 30-day session duration. Tokens are automatically refreshed when expired.
Token-Based Login (CI/CD)โ
envio-cloud login --token ghp_YOUR_TOKEN
Or using an environment variable:
export ENVIO_GITHUB_TOKEN=ghp_YOUR_TOKEN
envio-cloud login
Required GitHub token scopes: read:org, read:user, user:email.
Session Managementโ
envio-cloud token # Check current session
envio-cloud logout # Remove credentials
Context Managementโ
Like kubectl namespaces, envio-cloud lets you store default values for organisation and indexer so you don't have to pass them on every command. Flags (--org, --indexer) always override stored context.
# Set defaults
envio-cloud config set-org myorg
envio-cloud config set-indexer myindexer
# View current context
envio-cloud config get-context
# Commands now use defaults automatically
envio-cloud deployment status abc1234 # org and indexer from context
envio-cloud indexer settings get # both from context
# Flags override context
envio-cloud deployment status abc1234 --org other-org
# Clear stored context
envio-cloud config clear
Context is stored at ~/.envio-cloud/context.json. Resolution priority:
- Explicit positional arguments
--org/--indexerflags- Stored context
- GitHub login (organisation only)
| Command | Description |
|---|---|
config set-org | Set default organisation |
config set-indexer | Set default indexer |
config get-context | Show current defaults and where they come from |
config clear | Remove all stored defaults |
Commandsโ
Indexer Commandsโ
List Indexersโ
Lists indexers across every organisation you are a member of. Use --org to
scope to a single organisation. Requires authentication.
envio-cloud indexer list
envio-cloud indexer list --org myorg
envio-cloud indexer list --limit 10
envio-cloud indexer list -o json
| Flag | Description |
|---|---|
--org | Scope to a single organisation you belong to |
--limit | Limit number of results |
-o, --output | Output format (json) |
Get Indexer Detailsโ
envio-cloud indexer get [organisation]
envio-cloud indexer get hyperindex mjyoung114 -o json
envio-cloud indexer get hyperindex --org mjyoung114
Organisation can be omitted if set via context. Requires authentication โ you can only view indexers in organisations you are a member of.
Add an Indexerโ
envio-cloud indexer add --name my-indexer --repo my-repo
envio-cloud indexer add --name my-indexer --repo my-repo --branch main --tier development
envio-cloud indexer add --name my-indexer --repo my-repo --dry-run
| Flag | Description | Default |
|---|---|---|
-n, --name | Indexer name (required) | โ |
-r, --repo | Repository name (required) | โ |
-b, --branch | Deployment branch | envio |
-d, --root-dir | Root directory | ./ |
-c, --config-file | Config file path | config.yaml |
-t, --tier | Pricing tier | development |
-a, --access-type | Access type | public |
-e, --env-file | Environment file | โ |
--auto-deploy | Enable auto-deploy | true |
--dry-run | Preview without creating | โ |
-y, --yes | Skip confirmation prompts | โ |
Delete an Indexerโ
Permanently delete an indexer and all of its deployments. Requires typing the indexer name to confirm.
envio-cloud indexer delete myindexer myorg
envio-cloud indexer delete myindexer --org myorg
envio-cloud indexer delete myindexer myorg --yes # skip confirmation for CI/CD
This action cannot be undone. All deployments, data, and configuration for the indexer will be permanently removed.
View and Modify Settingsโ
# View current settings
envio-cloud indexer settings get myindexer myorg
# Modify settings (only specified flags are changed)
envio-cloud indexer settings set myindexer myorg --branch main
envio-cloud indexer settings set myindexer myorg --auto-deploy=false
envio-cloud indexer settings set myindexer myorg --config-file config.yaml --branch develop
| Flag (set) | Description |
|---|---|
--branch | Git branch for deployments |
--config-file | Path to config file |
--root-dir | Root directory within the repository |
--auto-deploy | Enable or disable auto-deploy on push |
--description | Indexer description |
--access-type | public or private |
Manage Environment Variablesโ
Environment variables can be managed from the CLI. All keys must be prefixed with ENVIO_. Changes take effect on the next deployment.
# List variables (values masked by default)
envio-cloud indexer env list myindexer myorg
envio-cloud indexer env list myindexer myorg --show-values
# Set one or more variables
envio-cloud indexer env set myindexer myorg ENVIO_API_KEY=abc123 ENVIO_DEBUG=true
# Remove a variable
envio-cloud indexer env delete myindexer myorg ENVIO_DEBUG
# Bulk import from a .env file
envio-cloud indexer env import myindexer myorg --file .env
The .env file format is one KEY=VALUE per line. Lines starting with # are ignored.
Configure IP Whitelistingโ
Restrict access to your indexer's GraphQL endpoint by IP address. Supports IPv4 addresses and CIDR notation.
# View current IP whitelist configuration
envio-cloud indexer security get myindexer myorg
# Add IPs to the whitelist
envio-cloud indexer security add-ip myindexer myorg 203.0.113.50
envio-cloud indexer security add-ip myindexer myorg 10.0.0.0/8
# Enable IP whitelisting (make sure to add IPs first)
envio-cloud indexer security enable myindexer myorg
# Disable IP whitelisting
envio-cloud indexer security disable myindexer myorg
# Restrict whitelisting to production deployments only
envio-cloud indexer security set-prod-only myindexer myorg true
# Remove an IP
envio-cloud indexer security remove-ip myindexer myorg 203.0.113.50
Add your IP addresses before enabling whitelisting โ otherwise you may lock yourself out. The CLI will warn you if you try to enable whitelisting with no IPs configured.
Deployment Commandsโ
All deployment commands accept arguments as [organisation]. Organisation and indexer can be omitted if set via envio-cloud config.
Deployment Metricsโ
envio-cloud deployment metrics [organisation]
envio-cloud deployment metrics hyperindex b3ead3a mjyoung114 --watch
envio-cloud deployment metrics hyperindex b3ead3a mjyoung114 -o json
No authentication required.
| Flag | Description |
|---|---|
--watch | Continuously poll for updates |
-o, --output | Output format (json) |
Deployment Statusโ
envio-cloud deployment status [organisation]
envio-cloud deployment status hyperindex b3ead3a mjyoung114 --watch-till-synced
| Flag | Description |
|---|---|
--watch-till-synced | Wait until deployment is fully synced |
Deployment Infoโ
envio-cloud deployment info [organisation]
Get Query Endpointโ
Returns the GraphQL query endpoint URL for a deployment. The endpoint is computed from deployment parameters and the cluster is resolved from the deployment tier via the API. Output is a bare URL, so it composes cleanly with shell scripting.
envio-cloud deployment endpoint [organisation]
envio-cloud deployment endpoint hyperindex b3ead3a mjyoung114
envio-cloud deployment endpoint hyperindex b3ead3a mjyoung114 -o json
Use the URL directly in a curl query:
curl "$(envio-cloud deployment endpoint hyperindex b3ead3a mjyoung114)" \
-H "Content-Type: application/json" \
-d '{"query": "{ _meta { chainMetadata { chainId } } }"}'
| Flag | Description |
|---|---|
--cluster | Override cluster (hyper, hypertierchicago, ip-projects, prodaws, staging) |
-o, --output | Output format (json) |
The ep alias is also available: envio-cloud deployment ep .
Promote a Deploymentโ
Promote a deployment to the production endpoint. Requires confirmation (y/N).
envio-cloud deployment promote [organisation]
envio-cloud deployment promote myindexer abc1234 myorg --yes
Delete a Deploymentโ
Permanently delete a deployment. Requires typing the indexer name to confirm.
envio-cloud deployment delete [organisation]
envio-cloud deployment delete myindexer abc1234 myorg --yes
This action cannot be undone. The deployment and its data will be permanently removed.
Restart a Deploymentโ
Restart a running deployment. There is a 10-minute cooldown between restarts.
envio-cloud deployment restart [organisation]
envio-cloud deployment restart myindexer abc1234 myorg --yes
Deployment Logsโ
Show build or runtime logs for a deployment.
envio-cloud deployment logs [organisation]
envio-cloud deployment logs myindexer abc1234 myorg --build
envio-cloud deployment logs myindexer abc1234 myorg --level error,warn
envio-cloud deployment logs myindexer abc1234 myorg --follow
| Flag | Description |
|---|---|
--build | Show build logs instead of runtime logs |
--level | Filter by log level (e.g., error,warn) |
--limit | Max number of log lines (default: 100) |
--follow | Poll for new logs every 10 seconds |
Repository Commandsโ
List Repositoriesโ
envio-cloud repos
envio-cloud repos -o json
Requires authentication.
Confirmation Promptsโ
Dangerous commands require confirmation before executing:
| Command | Confirmation type |
|---|---|
indexer delete | Type the indexer name |
deployment delete | Type the indexer name |
deployment promote | y/N prompt |
deployment restart | y/N prompt |
All prompts can be skipped with the --yes / -y flag for CI/CD usage.
Global Flagsโ
| Flag | Description |
|---|---|
--org | Override default organisation |
--indexer | Override default indexer |
-q, --quiet | Suppress informational messages |
-o, --output | Output format (json) |
--config | Specify config file path |
-h, --help | Display command help |
-v, --version | Show CLI version |
JSON Outputโ
All commands support JSON output via the -o json flag, making the CLI easy to integrate into scripts and automation pipelines.
Success response:
{"ok": true, "data": [ ... ]}
Error response:
{"ok": false, "error": "error message"}
Example with jq:
# Get event count for a deployment
envio-cloud deployment metrics hyperindex b3ead3a mjyoung114 -o json | jq '.data[].num_events_processed'
# List all indexer IDs in an org
envio-cloud indexer list --org enviodev -o json | jq -r '.data[].indexer_id'
Exit Codesโ
| Code | Meaning |
|---|---|
0 | Success |
1 | User error (invalid arguments, authentication required) |
2 | API or server error |
Related Documentationโ
- Envio Cloud Overview - Introduction to Envio Cloud
- Deploying Your Indexer - Step-by-step deployment guide via the dashboard
- Production Features - Tags, IP whitelisting, caching, and alerts
- Monitoring - Dashboard monitoring and alerts
- Envio CLI - Local development CLI reference
- npm package - Latest version and changelog
Hosted Service Billingโ
File: Hosted_Service/hosted-service-billing.mdx
Pricing & Billing
Envio offers flexible pricing options to meet the needs of projects at different stages of development.
Pricing Plansโ
Envio Cloud offers flexible pricing plans to match your project's needs, from free development environments to enterprise-grade dedicated hosting.
For the most up-to-date pricing information, detailed plan comparisons, and feature breakdowns, please visit our official Envio Pricing Page.
Available Plans:
| Plan | Price | Intended for |
|---|---|---|
| Development | Free | Testing, prototyping, and development. 30-day max lifespan, subject to fair usage limits |
| Production Small | Paid | Getting started with production deployments |
| Production Medium | Paid | Scaling your indexing operations with higher limits |
| Production Large | Paid | High-volume production workloads |
| Dedicated | Custom | Ultimate performance, isolated infrastructure, and custom SLAs |
What's included across paid plans:
- Higher event processing and storage limits (increases with each tier)
- Higher query rate limits on your GraphQL endpoint
- Effect API cache support for faster re-indexing (Medium plans and up)
- Monitoring, alerts, and deployment management
- Priority support (Dedicated plan)
The free development plan is intended for testing and development purposes only and should not be used as a production environment. Development plan deployments have a maximum life span of 30 days and Envio makes no guarantees regarding uptime, availability, or data persistence for deployments on the development plan. If you choose to use a development plan deployment in a production capacity, you do so entirely at your own risk. Envio assumes no liability or accountability for any downtime, data loss, or service interruptions that may occur on development plan deployments.
For detailed feature explanations, see our Features page. For deployment instructions, see our Deployment Guide. Not sure which option is right for your project? Book a call with our team to discuss your specific needs.
Self-Hosting Your Blockchain Indexerโ
File: Hosted_Service/self-hosting.md
This documentation page is actively being improved. Check back regularly for updates and additional information.
While Envio offers a fully managed cloud hosting solution via Envio Cloud, you may prefer to run your blockchain indexer on your own infrastructure. This guide covers everything you need to know about self-hosting Envio indexers.
We deeply appreciate users who choose Envio Cloud, as it directly supports our team and helps us continue developing and improving Envio's technology. If your use case allows for it, please consider the hosted option.
Why Self-Host?โ
Self-hosting gives you:
- Complete Control: Manage your own infrastructure and configurations
- Data Sovereignty: Keep all indexed data within your own systems
Self Hosting can be done with a variety of different infrastructure, tools and methods. The outline below is merely a starting point and does not offer a full production level solution. In some cases advanced knowledge of infrastructure, database management and networking may be required for a full production level solution.
Prerequisitesโ
Before self-hosting, ensure you have:
- Docker installed on your host machine
- Sufficient storage for blockchain data and the indexer database
- Adequate CPU and memory resources (requirements vary based on chains and indexing complexity)
- Required HyperSync and/or RPC endpoints
- Envio API token for HyperSync access (
ENVIO_API_TOKEN) โ required for continued access. See API Tokens.
Getting Startedโ
In general, if you want to self-host, you will likely use a Docker setup.
For a working example, check out the local-docker-example repository.
It contains a minimal Dockerfile and docker-compose.yaml that configure the Envio indexer together with PostgreSQL and Hasura.
Configuration Explainedโ
The compose file in that repository sets up three main services:
- PostgreSQL Database (
envio-postgres): Stores your indexed data - Hasura GraphQL Engine (
graphql-engine): Provides the GraphQL API for querying your data - Envio Indexer (
envio-indexer): The core indexing service that processes blockchain data
Environment Variablesโ
The configuration uses environment variables with sensible defaults. For production, you should customize:
- Envio API token (
ENVIO_API_TOKEN) - Database credentials (
ENVIO_PG_PASSWORD,ENVIO_PG_USER, etc.) - Hasura admin secret (
HASURA_GRAPHQL_ADMIN_SECRET) - Resource limits based on your workload requirements
Getting Helpโ
If you encounter issues with self-hosting:
- Check the Envio GitHub repository for known issues
- Join the Envio Discord community for community support
For most production use cases, we recommend using Envio Cloud to benefit from automatic scaling, monitoring, and maintenance.
Organisation Setupโ
File: Hosted_Service/organisation-setup.md
Use this guide to set up an organisation in Envio Cloud and grant access to your team.
Access Controlโ
Being a member of the GitHub organisation does not automatically grant access to the organisation in the Envio Cloud UI. Each member must be explicitly added by the organisation admin. If someone attempts to visit the organisation URL (e.g., https://envio.dev/app/) without being added, they'll see a "You are not a member of this team" message.
!Not a Member Error
Tutorial Op Bridge Depositsโ
File: Tutorials/tutorial-op-bridge-deposits.md
Introductionโ
This tutorial will guide you through indexing Optimism Standard Bridge deposits in under 5 minutes using Envio HyperIndex's no-code contract import feature.
The Optimism Standard Bridge enables the movement of ETH and ERC-20 tokens between Ethereum and Optimism. We'll index bridge deposit events by extracting the DepositFinalized logs emitted by the bridge contracts on both networks.
Prerequisitesโ
Before starting, ensure you have the following installed:
- Node.js (v22 or newer recommended)
- pnpm (recommended but not required)
- Docker Desktop (required to run the Envio indexer locally)
Note: Docker is specifically required to run your blockchain indexer locally. You can skip Docker installation if you plan only to use Envio Cloud.
Step 1: Initialize Your Indexerโ
- Open your terminal in an empty directory and run:
pnpx envio init
-
Name your indexer (we'll use "optimism-bridge-indexer" in this example):
-
Choose your preferred language (TypeScript, JavaScript, or ReScript):
Step 2: Import the Optimism Bridge Contractโ
-
Select Contract Import โ Block Explorer โ Optimism
-
Enter the Optimism bridge contract address:
0x4200000000000000000000000000000000000010 -
Select the
DepositFinalizedevent:- Navigate using arrow keys (โโ)
- Press spacebar to select the event
Tip: You can select multiple events to index simultaneously.
Step 3: Add the Ethereum Mainnet Bridge Contractโ
-
When prompted, select Add a new contract
-
Choose Block Explorer โ Ethereum Mainnet
-
Enter the Ethereum Mainnet gateway contract address:
0x99C9fc46f92E8a1c0deC1b1747d010903E884bE1 -
Select the
ETHDepositInitiatedevent -
When finished adding contracts, select I'm finished
Step 4: Start Your Indexerโ
- If you have any running indexers, stop them first:
pnpm envio stop
- Start your new indexer:
pnpm dev
This command:
- Starts the required Docker containers
- Sets up your database
- Launches the indexing process
- Opens the Hasura GraphQL interface
Step 5: Understanding the Generated Codeโ
Let's examine the key files that Envio generated:
1. config.yamlโ
This configuration file defines:
- Networks to index (Optimism and Ethereum Mainnet)
- Starting blocks for each network
- Contract addresses and ABIs
- Events to track
2. schema.graphqlโ
This schema defines the data structures for our selected events:
- Entity types based on event data
- Field types matching the event parameters
- Relationships between entities (if applicable)
3. src/handlersโ
This file contains the business logic for processing events:
- Functions that execute when events are detected
- Data transformation and storage logic
- Entity creation and relationship management
Step 6: Exploring Your Indexed Dataโ
Now you can interact with your indexed data:
Accessing Hasuraโ
- Open Hasura at http://localhost:8080
- When prompted, enter the admin password:
testing
Monitoring Indexing Progressโ
- Click the Data tab in the top navigation
- Find the
_events_sync_statetable to check indexing progress - Observe which blocks are currently being processed
Note: Thanks to Envio's HyperSync, indexing happens significantly faster than with standard RPC methods.
Querying Indexed Eventsโ
- Click the API tab
- Construct a GraphQL query to explore your data
Here's an example query to fetch the 10 largest bridge deposits:
query LargestDeposits {
DepositFinalized(limit: 10, order_by: { amount: desc }) {
l1Token
l2Token
from
to
amount
blockTimestamp
}
}
- Click the Play button to execute your query
Conclusionโ
Congratulations! You've successfully created an indexer for Optimism Bridge deposits across both Ethereum and Optimism networks.
What You've Learnedโ
- How to initialize a multi-network indexer using Envio
- How to import contracts from different blockchains
- How to query and explore indexed blockchain data
Next Stepsโ
- Try customizing the event handlers to add additional logic
- Create relationships between events on different networks
- Deploy your indexer to Envio Cloud
For more tutorials and advanced features, check out our documentation or watch our video walkthroughs on YouTube.
Tutorial Erc20 Token Transfersโ
File: Tutorials/tutorial-erc20-token-transfers.md
Introductionโ
In this tutorial, you'll learn how to index ERC20 token transfers on the Base network using Envio HyperIndex. By leveraging the no-code contract import feature, you'll be able to quickly analyze USDC transfer activity, including identifying the largest transfers.
We'll create an indexer that tracks all USDC token transfers on Base by extracting the Transfer events emitted by the USDC contract. The entire process takes less than 5 minutes to set up and start querying data.
Prerequisitesโ
Before starting, ensure you have the following installed:
- Node.js (v22 or newer recommended)
- pnpm (recommended but not required)
- Docker Desktop (required to run the Envio indexer locally)
Note: Docker is specifically required to run your blockchain indexer locally. You can skip Docker installation if you plan only to use Envio Cloud.
Step 1: Initialize Your Indexerโ
- Open your terminal in an empty directory and run:
pnpx envio init
-
Name your indexer (we'll use "usdc-base-transfer-indexer" in this example):
-
Choose your preferred language (TypeScript, JavaScript, or ReScript):
Step 2: Import the USDC Token Contractโ
-
Select Contract Import โ Block Explorer โ Base
-
Enter the USDC token contract address on Base:
0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913 -
Select the
Transferevent:- Navigate using arrow keys (โโ)
- Press spacebar to select the event
Tip: You can select multiple events to index simultaneously if needed.
- When finished adding contracts, select I'm finished
Step 3: Start Your Indexerโ
- If you have any running indexers, stop them first:
pnpm envio stop
Note: You can skip this step if this is your first time running an indexer.
- Start your new indexer:
pnpm dev
This command:
- Starts the required Docker containers
- Sets up your database
- Launches the indexing process
- Opens the Hasura GraphQL interface
Step 4: Understanding the Generated Codeโ
Let's examine the key files that Envio generated:
1. config.yamlโ
This configuration file defines:
- Network to index (Base)
- Starting block for indexing
- Contract address and ABI details
- Events to track (Transfer)
2. schema.graphqlโ
This schema defines the data structures for the Transfer event:
- Entity types based on event data
- Field types for sender, receiver, and amount
- Any relationships between entities
3. src/handlersโ
This directory contains the business logic for processing events:
- Functions that execute when Transfer events are detected
- Data transformation and storage logic
- Entity creation and relationship management
Step 5: Exploring Your Indexed Dataโ
Now you can interact with your indexed USDC transfer data:
Accessing Hasuraโ
- Open Hasura at http://localhost:8080
- When prompted, enter the admin password:
testing
Monitoring Indexing Progressโ
- Click the Data tab in the top navigation
- Find the
_events_sync_statetable to check indexing progress - Observe which blocks are currently being processed
Note: Thanks to Envio's HyperSync, you can index millions of USDC transfers in just minutes rather than hours or days with traditional methods.
Querying Indexed Eventsโ
- Click the API tab
- Construct a GraphQL query to explore your data
Here's an example query to fetch the 10 largest USDC transfers:
query LargestTransfers {
FiatTokenV2_2_Transfer(limit: 10, order_by: { value: desc }) {
from
to
value
blockTimestamp
}
}
- Click the Play button to execute your query
Conclusionโ
Congratulations! You've successfully created an indexer for USDC token transfers on Base. In just a few minutes, you've indexed over 3.6 million transfer events and can now query this data in real-time.
What You've Learnedโ
- How to initialize an indexer using Envio's contract import feature
- How to index ERC20 token transfers on the Base network
- How to query and analyze token transfer data using GraphQL
Next Stepsโ
- Try customizing the event handlers to add additional logic
- Create aggregated statistics about token transfers
- Add more tokens or events to your indexer
- Deploy your indexer to Envio Cloud
For more tutorials and advanced features, check out our documentation or watch our video walkthrough on YouTube.
Tutorial Indexing Fuelโ
File: Tutorials/tutorial-indexing-fuel.md
HyperIndex supports any EVM-compatible blockchain and the Fuel Network.
Blockchain indexers are vital to the success of any dApp. In this tutorial, we will create an Envio indexer for the Fuel dApp Sway Farm step by step.
Sway Farm is a simple farming game and for the sake of a real-world example, let's create the indexer for a leaderboard of all farmers ๐งโ๐พ
About Fuelโ
Fuel is an operating system purpose-built for Ethereum rollups. Fuel's unique architecture allows rollups to solve for PSI (parallelization, state minimized execution, interoperability). Powered by the FuelVM, Fuel aims to expand Ethereum's capability set without compromising security or decentralization.
Prerequisitesโ
Environment toolingโ
- Node.js (v22 or newer recommended)
- pnpm (recommended but not required)
- Docker Desktop (required to run the Envio indexer locally)
Note: Docker is specifically required to run your blockchain indexer locally. You can skip Docker installation if you plan only to use Envio Cloud.
Initialize the projectโ
Now that you have installed the prerequisite packages let's begin the practical steps of setting up the indexer.
Open your terminal in an empty directory and initialize a new indexer by running the command:
pnpx envio init
In the following prompt, choose the directory where you want to set up your project. The default is the current directory, but in the tutorial, I'll use the indexer name:
? Specify a folder name (ENTER to skip): sway-farm-indexer
Then, choose a language of your choice for the event handlers. TypeScript is the most popular one, so we'll stick with it:
? Which language would you like to use?
JavaScript
> TypeScript
ReScript
[โโ to move, enter to select, type to filter]
Next, we have the new prompt for a blockchain ecosystem. Previously Envio supported only EVM, but now it's possible to choose between Evm, Fuel and other VMs in the future:
? Choose blockchain ecosystem
Evm
> Fuel
[โโ to move, enter to select, type to filter]
In the following prompt, you can choose an initialization option. There's a Greeter template for Fuel, which is an excellent way to learn more about HyperIndex. But since we have an existing contract, the Contract Import option is the best way to create an indexer:
? Choose an initialization option
Template
> Contract Import
[โโ to move, enter to select, type to filter]
A separate Tutorial page provides more details about the
Greetertemplate.
Next it'll ask us for an ABI file. You can find it in the ./out/debug directory after building your Sway contract with forc build:
? What is the path to your json abi file? ./sway-farm/contract/out/debug/contract-abi.json
After the ABI file is provided, Envio parses all possible events you can use for indexing:
? Which events would you like to index?
> [x] NewPlayer
[x] PlantSeed
[x] SellItem
[x] InvalidError
[x] Harvest
[x] BuySeeds
[x] LevelUp
[โโ to move, space to select one, โ to all, โ to none, type to filter]
Let's select the events we want to index. I opened the code of the contract file and realized that for a leaderboard we need only events which update player information. Hence, I left only NewPlayer, LevelUp, and SellItem selected in the list. We'd want to index more events in real life, but this is enough for the tutorial.
? Which events would you like to index?
> [x] NewPlayer
[ ] PlantSeed
[x] SellItem
[ ] InvalidError
[ ] Harvest
[ ] BuySeeds
[x] LevelUp
[โโ to move, space to select one, โ to all, โ to none, type to filter]
๐ For the tutorial we only need to index
LOG_DATAreceipts, but you can also indexMint,Burn,TransferandCallreceipts. Read more about Supported Event Types.
Just a few simple questions left. Let's call our contract SwayFarm:
? What is the name of this contract? SwayFarm
Set an address for the deployed contract:
? What is the address of the contract? 0xf5b08689ada97df7fd2fbd67bee7dea6d219f117c1dc9345245da16fe4e99111
[Use the proxy address if your abi is a proxy implementation]
Finish the initialization process:
? Would you like to add another contract?
> I'm finished
Add a new address for same contract on same network
Add a new contract (with a different ABI)
[Current contract: SwayFarm, on network: Fuel]
If you see the following line, it means we are already halfway through ๐
Please run `cd sway-farm-indexer` to run the rest of the envio commands
Let's open the indexer in an IDE and start adjusting it for our farm ๐
Walk through initialized indexerโ
At this point, we should already have a working indexer. You can start it by running pnpm dev, which we cover in more detail later in the tutorial.
Everything is configured by modifying the 3 files below. Let's walk through each of them.
- config.yaml
Guide - schema.graphql
Guide - EventHandlers.*
Guide
(* depending on the language chosen for the indexer)
config.yamlโ
The config.yaml outlines the specifications for the indexer, including details such as network and contract specifications and the event information to be used in the indexing process.
name: sway-farm-indexer
ecosystem: fuel
chains:
- id: 0
start_block: 0
contracts:
- name: SwayFarm
address:
- "0xf5b08689ada97df7fd2fbd67bee7dea6d219f117c1dc9345245da16fe4e99111"
abi_file_path: abis/swayfarm-abi.json
events:
- name: SellItem
logId: "11192939610819626128"
- name: LevelUp
logId: "9956391856148830557"
- name: NewPlayer
logId: "169340015036328252"
In the tutorial, we don't need to adjust it in any way. But later you can modify the file and add more events for indexing.
As a nice to have, you can use a Sway struct name without specifying a logId, like this:
- name: SellItem
- name: LevelUp
- name: NewPlayer
schema.graphqlโ
The schema.graphql file serves as a representation of your application's data model. It defines entity types that directly correspond to database tables, and the event handlers you create are responsible for creating and updating records within those tables. Additionally, the GraphQL API is automatically generated based on the entity types specified in the schema.graphql file, to allow access to the indexed data.
๐ง A separate Guide page provides more details about the
schema.graphqlfile.
For the leaderboard, we need only one entity representing the player. Let's create it:
type Player {
id: ID!
farmingSkill: BigInt!
totalValueSold: BigInt!
}
We will use the user address as an ID. The fields farmingSkill and totalValueSold are u64 in Sway, so to safely map them to JavaScript value, we'll use BigInt.
EventHandlers.tsโ
The event handlers generated by contract import are quite simple and only add an entity to a DB when a related event is indexed.
/*
* Please refer to https://docs.envio.dev for a thorough guide on all Envio indexer features
*/
indexer.onEvent(
{ contract: "SwayFarm", event: "SellItem" },
async ({ event, context }) => {
const entity: Entity = {
id: `${event.chainId}_${event.block.height}_${event.logIndex}`,
};
context.SwayFarm_SellItem.set(entity);
},
);
Let's modify the handlers to update the Player entity instead. But before we start, we need to run pnpm codegen to generate utility code and types for the Player entity we've added.
pnpm codegen
It's time for a little bit of coding. The indexer is very simple; it requires us only to pass event data to an entity.
/**
Registers a handler that processes NewPlayer event
on the SwayFarm contract and stores the players in the DB
*/
indexer.onEvent(
{ contract: "SwayFarm", event: "NewPlayer" },
async ({ event, context }) => {
// Set the Player entity in the DB with the initial values
context.Player.set({
// The address in Sway is a union type of user Address and ContractID. Envio supports most of the Sway types, and the address value was decoded as a discriminated union 100% typesafe
id: event.params.address.payload.bits,
// Initial values taken from the contract logic
farmingSkill: 1n,
totalValueSold: 0n,
});
},
);
indexer.onEvent(
{ contract: "SwayFarm", event: "LevelUp" },
async ({ event, context }) => {
const playerInfo = event.params.player_info;
context.Player.set({
id: event.params.address.payload.bits,
farmingSkill: playerInfo.farming_skill,
totalValueSold: playerInfo.total_value_sold,
});
},
);
indexer.onEvent(
{ contract: "SwayFarm", event: "SellItem" },
async ({ event, context }) => {
const playerInfo = event.params.player_info;
context.Player.set({
id: event.params.address.payload.bits,
farmingSkill: playerInfo.farming_skill,
totalValueSold: playerInfo.total_value_sold,
});
},
);
Without overengineering, simply set the player data into the database. What's nice is that whenever your ABI or entities in graphql.schema change, Envio regenerates types and shows the compilation error.
๐ง You can find the indexer repo created during the tutorial on GitHub.
Starting the Indexerโ
๐ข Make sure you have docker open
The following commands will start the docker and create databases for indexed data. Make sure to re-run pnpm dev if you've made some changes.
pnpm dev
Nice, we indexed 1,721,352 blocks containing 58,784 events in 10 seconds, and they continue coming in.
View the indexed resultsโ
Let's check indexed players on the local Hasura server.
open http://localhost:8080
The Hasura admin-secret / password is testing, and the tables can be viewed in the data tab or queried from the playground.
Now, we can easily get the top 5 players, the number of inactive and active players, and the average sold value. What's left is a nice UI for the Sway Farm leaderboard, but that's not the tutorial's topic.
๐ง A separate Guide page provides more details about navigating Hasura.
Deploy the indexer to Envio Cloudโ
Once you have verified that the indexer is working for your contracts, then you are ready to deploy the indexer to Envio Cloud.
Deploying an indexer to Envio Cloud allows you to extract information via graphQL queries into your front-end or some back-end application.
Navigate to the Envio Cloud to start deploying your indexer and refer to this documentation for more information on deploying your indexer.
What next?โ
Once you have successfully finished the tutorial, you are ready to become a blockchain indexing wizard!
Join our Discord channel to make sure you catch all new releases.
Greeter Tutorialโ
File: Tutorials/greeter-tutorial.md
Introductionโ
This tutorial provides a step-by-step guide to indexing a simple Greeter smart contract deployed on multiple blockchains. You'll learn how to set up and run a multichain indexer using Envio's template system.
What is the Greeter Contract?โ
The Greeter contract is a straightforward smart contract that allows users to store greeting messages on the blockchain. For this tutorial, we'll be indexing instances of this contract deployed on both Polygon and Linea networks.
What You'll Buildโ
By the end of this tutorial, you'll have:
- A functioning multichain indexer that tracks greeting events
- The ability to query these events through a GraphQL endpoint
- Experience with Envio's core indexing functionality
Prerequisitesโ
Before starting, ensure you have the following installed:
- Node.js (v22 or newer recommended)
- pnpm (recommended but not required)
- Docker Desktop (required to run the Envio indexer locally)
Note: Docker is specifically required to run your blockchain indexer locally. You can skip Docker installation if you plan only to use Envio Cloud.
Step 1: Initialize Your Projectโ
First, let's create a new project using Envio's Greeter template:
- Open your terminal and run:
pnpx envio init
- When prompted for a directory, you can press Enter to use the current directory or specify another path:
? Set the directory: (.) .
- Choose your preferred programming language for event handlers:
? Which language would you like to use?
> JavaScript
TypeScript
ReScript
- Select the Template initialization option:
? Choose an initialization option
> Template
Contract Import
- Choose the Greeter template:
? Which template would you like to use?
> Greeter
Erc20
After completing these steps, Envio will generate all the necessary files for your indexer project.
Step 2: Understanding the Generated Filesโ
Let's examine the key files that were created:
config.yamlโ
This configuration file defines which networks and contracts to index:
# Partial example
chains:
- id: 137 # Polygon
# ... Polygon chain settings
contracts:
- name: Greeter
address: "0x9D02A17dE4E68545d3a58D3a20BbBE0399E05c9c"
# ... contract settings
- id: 59144 # Linea
# ... Linea chain settings
contracts:
- name: Greeter
address: "0xdEe21B97AB77a16B4b236F952e586cf8408CF32A"
# ... contract settings
schema.graphqlโ
This schema defines the data structures for the indexed events:
type Greeting {
id: ID!
user: String!
greeting: String!
blockNumber: Int!
blockTimestamp: Int!
transactionHash: String!
}
type User {
id: ID!
latestGreeting: String!
numberOfGreetings: Int!
greetings: [String!]!
}
src/handlers (or .ts/.res)โ
This file contains the logic to process events emitted by the Greeter contract.
Step 3: Start Your Indexerโ
Important: Make sure Docker Desktop is running before proceeding.
- Start the indexer with:
pnpm dev
This command:
- Launches Docker containers for the database and Hasura
- Sets up your local development environment
- Begins indexing data from the specified contracts
- Opens a terminal UI to monitor indexing progress
The indexer will retrieve data from both Polygon and Linea blockchains, starting from the blocks specified in your config.yaml file.
Step 4: Interact with the Contractsโ
To see your indexer in action, you can write new greetings to the blockchain:
For Polygon:โ
- Visit the contract on Polygonscan
- Connect your wallet
- Use the
setGreetingfunction to write a new greeting - Submit the transaction
For Linea:โ
- Visit the contract on Lineascan
- Connect your wallet
- Use the
setGreetingfunction to write a new greeting - Submit the transaction
Since this is a multichain example, you can interact with both contracts to see how Envio handles data from different blockchains simultaneously.
Step 5: Query the Indexed Dataโ
Now you can explore the data your indexer has captured:
- Open Hasura at http://localhost:8080
- When prompted for authentication, use the password:
testing - Navigate to the Data tab to browse the database tables
- Or use the API tab to write GraphQL queries
Example Queryโ
Try this query to see the latest greetings:
query GetGreetings {
Greeting(limit: 10, order_by: { blockTimestamp: desc }) {
id
user
greeting
blockNumber
blockTimestamp
transactionHash
}
}
Step 6: Deploy to Production (Optional)โ
When you're ready to move from local development to production:
- Visit Envio Cloud
- Follow the steps to deploy your indexer
- Get a production GraphQL endpoint for your application
For detailed deployment instructions, see the Envio Cloud documentation.
What You've Learnedโ
By completing this tutorial, you've learned:
- How to initialize an Envio project from a template
- How indexers process data from multiple blockchains
- How to query indexed data using GraphQL
- The basic structure of an Envio indexing project
Next Stepsโ
Now that you've mastered the basics, you can:
- Try the Contract Import feature to index any deployed contract
- Customize the event handlers to implement more complex indexing logic
- Add relationships between entities in your schema
- Explore Preload Optimization for faster handlers
- Create aggregated statistics from your indexed data
For more tutorials and examples, visit the Envio Documentation or join our Discord community for support.
Getting Price Data in Your Indexerโ
File: Tutorials/price-data.md
Introductionโ
Many blockchain applications require price data to calculate values such as:
- Historical token transfer values in USD
- Total value locked (TVL) in DeFi protocols over time
- Portfolio valuations at specific points in time
This tutorial explores three different approaches to incorporating price data into your Envio indexer, using a real-world example of tracking ETH deposits into a Uniswap V3 liquidity pool on the Blast blockchain.
TL;DR: The complete code for this tutorial is available in this GitHub repository.
What You'll Learnโ
In this tutorial, you'll:
- Compare three different methods for accessing token price data
- Analyze the tradeoffs between accuracy, decentralization, and performance
- Implement a multi-source price feed in an Envio indexer
- Build a practical example indexing Uniswap V3 liquidity events with price context
Price Data Methods Comparedโ
There are three primary methods to access price data within your indexer:
| Method | Description | Speed | Accuracy | Decentralization |
|---|---|---|---|---|
| Oracles | On-chain price feeds (e.g., API3, Chainlink) | Fast | Medium | Medium |
| DEX Pools | Swap events from decentralized exchanges | Fast | Medium-High | High |
| Off-chain APIs | External services (e.g., CoinGecko) | Slow | High | Low |
Let's explore each method in detail.
Method 1: Using Oracle Price Feedsโ
Oracle networks provide on-chain price data through specialized smart contracts. For this tutorial, we'll use API3 price feeds on Blast.
How Oracles Workโ
Oracle services like API3 maintain a network of data providers that push price updates to on-chain contracts. These updates typically occur:
- At regular time intervals
- When price deviations exceed a predefined threshold (e.g., 1%)
- When manually triggered by network participants
Finding the Right Oracle Feedโ
To locate the ETH/USD price feed using API3 on Blast:
-
Identify the API3 contract address:
0x709944a48cAf83535e43471680fDA4905FB3920a -
Find the data feed ID for ETH/USD:
- The dAPI name "ETH/USD" as bytes32:
0x4554482f55534400000000000000000000000000000000000000000000000000 - Using the
dapiNameToDataFeedIdfunction, this maps to0x3efb3990846102448c3ee2e47d22f1e5433cd45fa56901abe7ab3ffa054f70b5
- The dAPI name "ETH/USD" as bytes32:
-
Monitor the
UpdatedBeaconSetWithBeaconsevents with this data feed ID to get price updates
Oracle Advantages and Limitationsโ
Advantages:
- Fast indexing (no external API calls required)
- Moderate decentralization
- Generally reliable data
Limitations:
- Updates only on significant price changes
- Limited token coverage (mainly high-liquidity pairs)
- Minor accuracy tradeoffs
Method 2: Using DEX Pool Swap Eventsโ
Decentralized exchanges like Uniswap provide price data through swap events. We'll use the USDB/WETH pool on Blast to derive ETH pricing.
Locating the Right DEX Poolโ
First, we need to find the specific Uniswap V3 pool for USDB/WETH:
const usdb = "0x4300000000000000000000000000000000000003";
const weth = "0x4300000000000000000000000000000000000004";
const factoryAddress = "0x792edAdE80af5fC680d96a2eD80A44247D2Cf6Fd";
const factoryAbi = parseAbi([
"function getPool( address tokenA, address tokenB, uint24 fee ) external view returns (address pool)",
]);
const providerUrl = "https://rpc.ankr.com/blast";
const poolBips = 3000; // 0.3%. This is measured in hundredths of a bip
const client = createPublicClient({
chain: blast,
transport: http(providerUrl),
});
const factoryContract = getContract({
abi: factoryAbi,
address: factoryAddress,
client: client,
});
(async () => {
const poolAddress = await factoryContract.read.getPool([
usdb,
weth,
poolBips,
]);
console.log(poolAddress);
})();
Tip: You can also manually find the pool address using the
getPoolfunction on a block explorer.
Running this code reveals the USDB/WETH pool is at 0xf52B4b69123CbcF07798AE8265642793b2E8990C.
Getting Price Data From Swap Eventsโ
Uniswap V3 emits Swap events containing price information in the sqrtPriceX96 field. To convert this to a price, we'll use a formula in our event handler.
DEX Advantages and Limitationsโ
Advantages:
- Very decentralized
- High update frequency
- Wide token coverage
Limitations:
- Susceptible to price impact and manipulation (especially in low-liquidity pools)
- Requires extra calculations to derive prices
- May require multiple pools for cross-pair calculations
Method 3: Using Off-chain APIsโ
External price APIs like CoinGecko provide comprehensive token price data but require HTTP calls from your indexer.
Making API Requestsโ
Here's a simple function to fetch historical ETH prices from CoinGecko:
const COIN_GECKO_API_KEY = process.env.COIN_GECKO_API_KEY;
async function fetchEthPriceFromUnix(
unix: number,
token = "ethereum"
): Promise {
// convert unix to date dd-mm-yyyy
const _date = new Date(unix * 1000);
const date = _date.toISOString().slice(0, 10).split("-").reverse().join("-");
return fetchEthPrice(date.slice(0, 10), token);
}
async function fetchEthPrice(
date: string,
token = "ethereum"
): Promise {
const options = {
method: "GET",
headers: {
accept: "application/json",
"x-cg-demo-api-key": COIN_GECKO_API_KEY,
},
};
return fetch(
`https://api.coingecko.com/api/v3/coins/${token}/history?date=${date}&localization=false`,
options as any
)
.then((res) => res.json())
.then((res: any) => {
const usdPrice = res.market_data.current_price.usd;
console.log(`ETH price on ${date}: ${usdPrice}`);
return usdPrice;
})
.catch((err) => console.error(err));
}
export default fetchEthPriceFromUnix;
Note: The free CoinGecko API only provides daily price data (at 00:00 UTC), not block-by-block precision. For production use, consider a paid API with more granular historical data.
Off-chain API Advantages and Limitationsโ
Advantages:
- Highest accuracy (with paid APIs)
- Most comprehensive token coverage
- No susceptibility to on-chain manipulation
Limitations:
- Significantly slows indexing speed due to API calls
- Centralized data source
- May require paid subscriptions for full functionality
Building a Multi-Source Price Feed Indexerโ
Now let's build an indexer that compares all three methods when tracking Uniswap V3 liquidity pool deposits.
Step 1: Initialize Your Indexerโ
Create a new Envio indexer project:
pnpx envio init
Step 2: Configure Your Indexerโ
Edit your config.yaml file to track both the API3 oracle and the Uniswap V3 pool:
# yaml-language-server: $schema=./node_modules/envio/evm.schema.json
name: envio-indexer
chains:
- id: 81457
start_block: 11000000
contracts:
- name: Api3ServerV1
address:
- "0x709944a48cAf83535e43471680fDA4905FB3920a"
events:
- event: UpdatedBeaconSetWithBeacons(bytes32 indexed beaconSetId, int224 value, uint32 timestamp)
- name: UniswapV3Pool
address:
- "0xf52B4b69123CbcF07798AE8265642793b2E8990C"
events:
- event: Swap(address indexed sender, address indexed recipient, int256 amount0, int256 amount1, uint160 sqrtPriceX96, uint128 liquidity, int24 tick)
- event: Mint(address sender, address indexed owner, int24 indexed tickLower, int24 indexed tickUpper, uint128 amount, uint256 amount0, uint256 amount1)
field_selection:
transaction_fields:
- "hash"
Important: The
field_selectionsection is needed to include transaction hashes in your indexed data.
Step 3: Define Your Schemaโ
Create a schema that captures price data from all three sources:
type OraclePoolPrice {
id: ID!
value: BigInt!
timestamp: BigInt!
block: Int!
}
type UniswapV3PoolPrice {
id: ID!
sqrtPriceX96: BigInt!
timestamp: Int!
block: Int!
}
type EthDeposited {
id: ID!
timestamp: Int!
block: Int!
oraclePrice: Float!
poolPrice: Float!
offChainPrice: Float!
offchainOracleDiff: Float!
depositedPool: Float!
depositedOffchain: Float!
depositedOracle: Float!
txHash: String!
}
Step 4: Implement Event Handlersโ
Create event handlers to process data from all three sources:
let latestOraclePrice = 0;
let latestPoolPrice = 0;
indexer.onEvent(
{ contract: "Api3ServerV1", event: "UpdatedBeaconSetWithBeacons" },
async ({ event, context }) => {
// Filter out the beacon set for the ETH/USD price
if (
event.params.beaconSetId !=
"0x3efb3990846102448c3ee2e47d22f1e5433cd45fa56901abe7ab3ffa054f70b5"
) {
return;
}
const entity: Entity = {
id: `${event.chainId}-${event.block.number}-${event.logIndex}`,
value: event.params.value,
timestamp: event.params.timestamp,
block: event.block.number,
};
latestOraclePrice = Number(event.params.value) / Number(10 ** 18);
context.OraclePoolPrice.set(entity);
},
);
indexer.onEvent(
{ contract: "UniswapV3Pool", event: "Swap" },
async ({ event, context }) => {
const entity: Entity = {
id: `${event.chainId}-${event.block.number}-${event.logIndex}`,
sqrtPriceX96: event.params.sqrtPriceX96,
timestamp: event.block.timestamp,
block: event.block.number,
};
latestPoolPrice = Number(
(2n ** 192n) /
(BigInt(event.params.sqrtPriceX96) * BigInt(event.params.sqrtPriceX96))
);
context.UniswapV3PoolPrice.set(entity);
},
);
indexer.onEvent(
{ contract: "UniswapV3Pool", event: "Mint" },
async ({ event, context }) => {
const offChainPrice = await fetchEthPriceFromUnix(event.block.timestamp);
const ethDepositedUsdPool =
(latestPoolPrice * Number(event.params.amount1)) / 10 ** 18;
const ethDepositedUsdOffchain =
(offChainPrice * Number(event.params.amount1)) / 10 ** 18;
const ethDepositedUsdOracle =
(latestOraclePrice * Number(event.params.amount1)) / 10 ** 18;
const ethDeposited: Entity = {
id: `${event.chainId}-${event.block.number}-${event.logIndex}`,
timestamp: event.block.timestamp,
block: event.block.number,
oraclePrice: round(latestOraclePrice),
poolPrice: round(latestPoolPrice),
offChainPrice: round(offChainPrice),
depositedPool: round(ethDepositedUsdPool),
depositedOffchain: round(ethDepositedUsdOffchain),
depositedOracle: round(ethDepositedUsdOracle),
offchainOracleDiff: round(
((ethDepositedUsdOffchain - ethDepositedUsdOracle) /
ethDepositedUsdOffchain) *
100
),
txHash: event.transaction.hash,
};
context.EthDeposited.set(ethDeposited);
},
);
function round(value: number) {
return Math.round(value * 100) / 100;
}
Step 5: Run Your Indexerโ
Start your indexer with:
pnpm dev
This will begin indexing data from block 11,000,000 on Blast.
Step 6: Analyze the Resultsโ
After running your indexer, you can query the data in Hasura to compare the three price data sources:
query ComparePrices {
EthDeposited(order_by: { block: desc }, limit: 10) {
block
timestamp
oraclePrice
poolPrice
offChainPrice
depositedPool
depositedOffchain
depositedOracle
offchainOracleDiff
txHash
}
}
Results Analysisโ
When comparing our three price data sources, we found:
Looking at the offchainOracleDiff column, we can see that oracle and off-chain prices typically align closely but can deviate by as much as 17.98% in some cases.
For the highlighted transaction (0xe7e79ddf29ed2f0ea8cb5bb4ffdab1ea23d0a3a0a57cacfa875f0d15768ba37d), we can compare our calculated values:
- Actual value (from block explorer): $2,358.27
- DEX pool value (
depositedPool): $2,117.07 - Off-chain API value (
depositedOffchain): $2,156.15
This demonstrates that even the most accurate methods have limitations.
Conclusion: Choosing the Right Methodโ
Based on our analysis, here are some recommendations for choosing a price data method:
Use Oracle or DEX Pools when:โ
- Indexing speed is critical
- Absolute precision isn't required
- You're working with high-liquidity tokens
Use Off-chain APIs when:โ
- Price accuracy is paramount
- Indexing speed is less important
- You can implement effective caching
For maximum accuracy while maintaining performance:โ
- Combine multiple methods and aggregate results
- Use high-volume DEX pools on major networks
- Cache API results to avoid redundant calls
Next Stepsโ
To further enhance your price data indexing:
- Implement caching for off-chain API calls
- Cross-reference multiple DEX pools for better accuracy
- Consider time-weighted average prices (TWAP) instead of spot prices
- Use multichain indexing to access higher-liquidity pools on major networks
By carefully choosing and implementing the right price data strategy, you can build robust indexers that provide accurate financial data for your blockchain applications.
Scaffold-Eth-2 Envio Extensionโ
File: Tutorials/tutorial-scaffold-eth-2.md
Introductionโ
The Scaffold-ETH 2 Envio extension makes indexing your deployed smart contracts as simple as possible. Generate a boilerplate indexer for your deployed contracts with a single click and start indexing their events immediately.
With this extension, you get:
- ๐ Automatic indexer generation from your deployed contracts
- ๐ Status dashboard with links to Envio metrics and database
- ๐ One-click regeneration to update the indexer when you deploy new contracts
- ๐ GraphQL API for querying your indexed blockchain data
Prerequisitesโ
Before starting, ensure you have the following installed:
- Node.js v20 (v20 or newer required)
- pnpm (for Envio indexer)
- Docker Desktop (required to run the Envio indexer locally)
- Yarn (for Scaffold-ETH)
Step 1: Create a New Scaffold-ETH 2 Project with Envio Extensionโ
To create a new Scaffold-ETH 2 project with the Envio extension already integrated:
npx create-eth@latest -e enviodev/scaffold-eth-2-extension
Step 2: Start the Local Blockchainโ
Navigate to your project directory and start the local blockchain:
cd your-project-name
yarn chain
This will start a local blockchain node for development.
Step 3: Deploy Your Contractsโ
In a new terminal window, navigate to your project directory and deploy the default smart contracts:
cd your-project-name
yarn deploy
This will deploy the default contracts to the local blockchain. This step is optional and can also be done once you've created your own smart contracts and deployed them using yarn deploy.
Step 4: Start Scaffold-ETH Frontendโ
From your project directory, start the Scaffold-ETH frontend:
yarn start
This will start the Scaffold-ETH frontend at http://localhost:3000.
Step 5: Generate the Indexerโ
Navigate to the Envio page in your Scaffold-ETH frontend at http://localhost:3000/envio and click the "Generate" button. This should only be done once you've created a smart contract and ran yarn deploy. This will create the boilerplate indexer from your deployed contracts.
The Envio page also includes a helpful "How to Use" section with step-by-step instructions.
Step 6: Start the Indexerโ
Navigate to the Envio package directory and start the indexer:
cd packages/envio
pnpm dev
This will begin indexing your contract events.
Regenerating the Indexerโ
When you deploy new contracts or make changes to existing ones, you'll need to regenerate the indexer:
Via Frontend Dashboardโ
- Go to the Envio page at
http://localhost:3000/envio - Click "Generate" to regenerate the boilerplate indexer
Via Command Lineโ
cd packages/envio
pnpm update
pnpm codegen
Note: Regenerating will overwrite any custom handlers, config, and schema changes, creating a fresh boilerplate indexer based on your deployed contracts. After regenerating, you'll need to stop the running indexer (Ctrl+C) and restart it with
pnpm devfor the changes to take effect.
Dynamic Contractsโ
File: Advanced/dynamic-contracts.md
Introductionโ
Many blockchain systems use factory patterns where new contracts are created dynamically. Common examples include:
- DEXes like Uniswap where each trading pair creates a new contract
- NFT platforms that deploy new collection contracts
- Lending protocols that create new markets as isolated contracts
When indexing these systems, you need a way to discover and track these dynamically created contracts. Envio provides powerful tools to handle this use case.
Contract Registration Handlerโ
Instead of a template based approach, we've introduced a contractRegister handler that can be added to any event.
This allows you to easily:
- Register contracts from any event handler.
- Use conditions and any logic you want to register contracts.
- Have nested factories which are registered by other factories.
indexer.contractRegister(
{ contract: "", event: "" },
({ event, context }) => {
context.chain..add();
},
);
Example: NFT Factory Patternโ
Let's look at a complete example using an NFT factory pattern.
Scenarioโ
NftFactorycontract creates newSimpleNftcontracts- We want to index events from all NFTs created by this factory
- Each time a new NFT is created, the factory emits a
SimpleNftCreatedevent
1. Configure Your Contracts in config.yamlโ
name: nftindexer
description: NFT Factory
chains:
- id: 1337
start_block: 0
contracts:
- name: NftFactory
abi_file_path: abis/NftFactory.json
address: "0x4675a6B115329294e0518A2B7cC12B70987895C4" # Factory address is known
events:
- event: SimpleNftCreated (string name, string symbol, uint256 maxSupply, address contractAddress)
- name: SimpleNft
abi_file_path: abis/SimpleNft.json
# No address field - we'll discover these addresses from events
events:
- event: Transfer (address from, address to, uint256 tokenId)
Note that:
- The
NftFactorycontract has a known address specified in the config - The
SimpleNftcontract has no address, as we'll register instances dynamically
2. Create the Contract Registration Handlerโ
In your src/handlers/.ts file:
// Register SimpleNft contracts whenever they're created by the factory
indexer.contractRegister(
{ contract: "NftFactory", event: "SimpleNftCreated" },
({ event, context }) => {
// Register the new NFT contract using its address from the event
context.chain.SimpleNft.add(event.params.contractAddress);
context.log.info(
`Registered new SimpleNft at ${event.params.contractAddress}`
);
},
);
// Handle Transfer events from all SimpleNft contracts
indexer.onEvent(
{ contract: "SimpleNft", event: "Transfer" },
async ({ event, context }) => {
// Your event handling logic here
context.log.info(
`NFT Transfer at ${event.srcAddress} - Token ID: ${event.params.tokenId}`
);
// Example: Store transfer information in the database
// ...
},
);
Async Contract Registerโ
As of version 2.21, you can use async contract registration.
This is a unique feature of Envio that allows you to perform an external call to determine the address of the contract to register.
indexer.contractRegister(
{ contract: "NftFactory", event: "SimpleNftCreated" },
async ({ event, context }) => {
const version = await getContractVersion(event.params.contractAddress);
if (version === "v2") {
context.chain.SimpleNftV2.add(event.params.contractAddress);
} else {
context.chain.SimpleNft.add(event.params.contractAddress);
}
},
);
Coming from TheGraph?โ
If you're migrating from a subgraph that uses data source templates (DataSource.create()), the equivalent in Envio is the contractRegister handler.
| TheGraph | Envio (HyperIndex) |
|---|---|
Define a template in subgraph.yaml | Define the contract in config.yaml without an address |
Call MyTemplate.create(address) in a mapping | Call context.chain.MyContract.add(address) in a contractRegister handler |
| Templates are triggered from other mappings | contractRegister runs before the event handler, on any event |
The key difference is that Envio's contractRegister is more flexible โ you can add conditional logic, perform async calls, and register contracts from any event, not just from a dedicated factory mapping.
For a step-by-step migration guide, see Migrating from a Subgraph.
When to Use Dynamic Contract Registrationโ
Use dynamic contract registration when:
- Your system includes factory contracts that deploy new contracts over time
- You want to index events from all instances of a particular contract type
- The addresses of these contracts aren't known at the time you create your indexer
Important Notesโ
-
Block Coverage: When a dynamic contract is registered, Envio will index all events from that contract in the same block where it was created, even if those events happened in transactions before the registration event. This is particularly useful for contracts that emit events during their construction.
-
Handler Organization: You can register contracts from any event handler. For example, you might register a token contract when you see it being added to a registry, not just when it's created.
-
Pre-registration: Pre-registration was a recommended mode in early V2 to optimize performance. The
preRegisterDynamicContractsoption has been removed entirely in V3 โ the default registration path is now the fastest, so no flag is needed.
Debugging Tipsโ
- Use logging in your
contractRegisterfunction to confirm contracts are being registered. - If you're not seeing events from your dynamic contracts, verify they're being properly registered in database.
For more information on writing event handlers, see the Event Handlers Guide.
Wildcard Indexingโ
File: Advanced/wildcard-indexing.mdx
Wildcard indexing is a feature that allows you to index all events matching a specified event signature without requiring the contract address from which the event was emitted. This is useful in cases such as indexing contracts deployed through factories, where the factory contract does not emit any events upon contract creation. It also enables indexing events from all contracts implementing a standard (e.g. all ERC20 transfers).
Wildcard Indexing is supported on HyperSync, HyperFuel, and RPC data sources.
Index all ERC20 transfersโ
As an example, let's say we want to index all ERC20 Transfer events. Start with a config.yaml file:
name: transefer-indexer
chains:
- id: 1
start_block: 0
contracts:
- name: ERC20
events:
- event: Transfer(address indexed from, address indexed to, uint256 value)
Let's also define some entities in schema.graphql file, so our handlers can store the processed data:
type Transfer {
id: ID!
from: String!
to: String!
}
And the last bit is to register an event handler in the src/handlers. Note how we pass the wildcard: true option to enable wildcard indexing:
indexer.onEvent(
{ contract: "ERC20", event: "Transfer", wildcard: true },
async ({ event, context }) => {
context.Transfer.set({
id: `${event.chainId}_${event.block.number}_${event.logIndex}`,
from: event.params.from,
to: event.params.to,
});
},
);
After running your indexer with pnpm dev you will have all ERC20 Transfer events indexed, regardless of the contract address from which the event was emitted.
Topic Filteringโ
Indexing all ERC20 Transfer events can be noisy. Use Topic Filtering to keep only the events you need.
When registering an event handler or a contract registration handler, provide the where option (formerly eventFilters in V2). Filter by each indexed event parameter by returning { params: [...] }.
Let's say you only want to index Mint events where the from address is equal to ZERO_ADDRESS:
const ZERO_ADDRESS = "0x0000000000000000000000000000000000000000";
indexer.onEvent(
{
contract: "ERC20",
event: "Transfer",
wildcard: true,
where: () => ({ params: [{ from: ZERO_ADDRESS }] }),
},
async ({ event, context }) => {
//... your handler logic
},
);
Multiple Filtersโ
If you want to index both Mint and Burn events you can provide multiple filters as an array. Also, every parameter can accept an array to filter by multiple possible values. We'll use it to filter by a group of whitelisted addresses in the example below:
const ZERO_ADDRESS = "0x0000000000000000000000000000000000000000";
const WHITELISTED_ADDRESSES = [
"0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266",
"0x70997970C51812dc3A010C7d01b50e0d17dc79C8",
"0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC",
];
indexer.onEvent(
{
contract: "ERC20",
event: "Transfer",
wildcard: true,
where: () => ({
params: [
{ from: ZERO_ADDRESS, to: WHITELISTED_ADDRESSES },
{ from: WHITELISTED_ADDRESSES, to: ZERO_ADDRESS },
],
}),
},
async ({ event, context }) => {
//... your handler logic
},
);
Different Filters per Chainโ
For Multichain Indexers the where callback receives { chain } and you can read chain.id to filter by different values per chain:
const ZERO_ADDRESS = "0x0000000000000000000000000000000000000000";
const WHITELISTED_ADDRESSES = {
1: ["0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266"],
137: [
"0x70997970C51812dc3A010C7d01b50e0d17dc79C8",
"0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC",
],
};
indexer.onEvent(
{
contract: "ERC20",
event: "Transfer",
wildcard: true,
where: ({ chain }) => ({
params: [
{ from: ZERO_ADDRESS, to: WHITELISTED_ADDRESSES[chain.id] },
{ from: WHITELISTED_ADDRESSES[chain.id], to: ZERO_ADDRESS },
],
}),
},
async ({ event, context }) => {
//... your handler logic
},
);
Index all ERC20 transfers to your Contractโ
Besides chain.id you can also read the contract's configured (and dynamically registered) addresses from chain..addresses.
For example, if you have a Safe contract, you can index all ERC20 transfers sent specifically to/from your Safe contracts. The where callback can read chain.Safe.addresses, so we need to define the Transfer event on the Safe contract:
name: locker
chains:
- id: 1
start_block: 0
contracts:
- name: Safe
events:
- event: Transfer(address indexed from, address indexed to, uint256 value)
address:
- "0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266"
- "0x70997970C51812dc3A010C7d01b50e0d17dc79C8"
- "0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC"
indexer.onEvent(
{
contract: "Safe",
event: "Transfer",
wildcard: true,
where: ({ chain }) => ({
params: [
{ from: chain.Safe.addresses },
{ to: chain.Safe.addresses },
],
}),
},
async ({ event, context }) => {},
);
This example is not much different from using a WHITELISTED_ADDRESSES constant, but this becomes much more powerful when the Safe contract addresses are registered dynamically by a factory contract:
name: locker
chains:
- id: 1
start_block: 0
contracts:
- name: SafeRegistry
events:
- event: NewSafe(address safe)
address:
- "0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266"
- name: Safe
events:
- event: Transfer(address indexed from, address indexed to, uint256 value)
indexer.contractRegister(
{ contract: "SafeRegistry", event: "NewSafe" },
async ({ event, context }) => {
context.chain.Safe.add(event.params.safe);
},
);
indexer.onEvent(
{
contract: "Safe",
event: "Transfer",
wildcard: true,
where: ({ chain }) => ({
params: [
{ from: chain.Safe.addresses },
{ to: chain.Safe.addresses },
],
}),
},
async ({ event, context }) => {},
);
Assert ERC20 Transfers in Handlerโ
After you got all ERC20 Transfers relevant to your contracts, you can additionally filter them in the handler. For example, to get only USDC transfers:
const USDC_ADDRESS = {
84532: "0x036CbD53842c5426634e7929541eC2318f3dCF7e",
11155111: "0x1c7D4B196Cb0C7B01d743Fbc6116a902379C7238",
};
indexer.onEvent(
{
contract: "Safe",
event: "Transfer",
wildcard: true,
where: ({ chain }) => ({
params: [
{ from: chain.Safe.addresses },
{ to: chain.Safe.addresses },
],
}),
},
async ({ event, context }) => {
// Filter and store only the USDC transfers that involve a Safe address
if (event.srcAddress === USDC_ADDRESS[event.chainId]) {
context.Transfer.set({
id: `${event.chainId}_${event.block.number}_${event.logIndex}`,
from: event.params.from,
to: event.params.to,
});
}
},
);
Contract Register Exampleโ
The same where option can be applied to indexer.contractRegister. Here is an example where we only register Uniswap pools that contain DAI token:
const DAI_ADDRESS = "0x6B175474E89094C44Da98b954EedeAC495271d0F";
indexer.contractRegister(
{
contract: "UniV3Factory",
event: "PoolCreated",
where: () => ({
params: [{ token0: DAI_ADDRESS }, { token1: DAI_ADDRESS }],
}),
},
async ({ event, context }) => {
const poolAddress = event.params.pool;
context.chain.UniV3Pool.add(poolAddress);
},
);
Limitationsโ
-
For any given chain, only one event of a given signature can be indexed using wildcard indexing. This means that if you have multiple contract definitions in your config that contain the same event signature, only one of them is allowed to be set to
wildcard: true. -
Either the
contractRegisteror theonEventregistration for a given event can take awhereoption, but not both. -
The RPC data source currently supports Topic Filtering only applied to a single wildcard event.
Preload Optimizationโ
File: Advanced/preload-optimization.md
Important! Preload optimization makes your handlers run twice.
In HyperIndex V3, preload optimization is always on โ there is no flag to enable or disable it.
This optimization enables HyperIndex to efficiently preload entities used by handlers through batched database queries, while ensuring events are processed synchronously in their original order. When combined with the Effect API for external calls, this feature delivers performance improvements of multiple orders of magnitude compared to other indexing solutions.
Configureโ
Nothing to configure. Previously, V2 required the preload_handlers: true flag in config.yaml. In V3 the flag has been removed and the optimization is always active. If your project still has preload_handlers: in config.yaml, delete it โ V3 will reject the field.
Why Preload?โ
To ensure reliable data, HyperIndex guarantees that all events will be processed in the same order as they occurred on-chain.
This guarantee is crucial as it allows you to build indexers that depend on the sequential order of events.
However, this leads to a challenge: Handlers must run one at a time, sequentially for each event. Any asynchronous operations will block the entire process.
To solve this, we introduced Preload Optimization.
It combines in-memory storage, batching, deduplication, and the Effect API to parallelize asynchronous operations across batches of events.
How It Works?โ
With Preload Optimization handlers run twice per event:
- First Run (Preload Phase): All event handlers run concurrently for the whole batch of events. During the phase all DB write operations are skipped and only DB read operations and external calls are performed.
- Second Run (Processing Phase): Each event handler runs sequentially in the on-chain order. During the phase it'll get the data from the in-memory store, reflecting changes made by previously processed events.
This double execution pattern ensures that entities created by earlier events in the batch are available to later events.
The Database I/O Problemโ
Consider this common pattern of getting entities in event handlers:
indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event, context }) => {
const sender = await context.Account.get(event.params.from);
const receiver = await context.Account.get(event.params.to);
// Process the transfer...
},
);
Without Preload Optimization: If you're processing 5,000 transfer events, each with unique from and to addresses, this results in 10,000 total database roundtripsโone for each sender and receiver lookup (2 per event ร 5,000 events). This creates a significant bottleneck that slows down your entire indexing process.
With Preload Optimization: During the Preload Phase, all 5,000 events are processed in parallel. HyperIndex batches database reads that occur simultaneously into single database queries - one query for sender lookups and one for receiver lookups. The loaded accounts are cached in memory. After the Preload Phase completes, the second processing phase begins. This phase runs handlers sequentially in on-chain order, but instead of making database calls, it retrieves the data from the in-memory cache.
For our example of 5,000 transfer events, this optimization reduces database roundtrips from 10,000 calls to just 2!
Optimizing for Concurrencyโ
You can further optimize performance by requesting multiple entities concurrently:
indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event, context }) => {
// Request sender and receiver concurrently for maximum efficiency
const [sender, receiver] = await Promise.all([
context.Account.get(event.params.from),
context.Account.get(event.params.to),
]);
// Process the transfer...
},
);
This approach can reduce the database roundtrips to just 1 for the entire batch of events!
The External Calls Problemโ
Let's say you want to populate your indexer with offchain data:
indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event, context }) => {
// Without Preload: Blocking external calls
const metadata = await fetch(
`https://api.example.com/metadata/${event.params.from}`
);
// Process the transfer...
},
);
Without Preload Optimization: If you're processing 5,000 transfer events, each with an external call, this results in 5,000 sequential external callsโeach waiting for the previous one to complete. This can turn a fast indexing process into a slow, sequential crawl.
With Preload Optimization: Since handlers run twice for each event, making direct external calls can be problematic. The Effect API provides a solution. During the Preload Phase, it batches all external calls and runs them in parallel. Then during the Processing Phase, it runs the handlers sequentially, retrieving the already requested data from the in-memory store.
const fetchMetadata = createEffect(
{
name: "fetchMetadata",
input: {
from: S.string,
},
output: {
decimals: S.number,
symbol: S.string,
},
rateLimit: {
calls: 5,
per: "second",
},
},
async ({ input }) => {
const metadata = await fetch(
`https://api.example.com/metadata/${input.from}`
);
return metadata;
}
);
indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event, context }) => {
// With Preload: Performs the call in parallel
const metadata = await context.effect(fetchMetadata, {
from: event.params.from,
});
// Process the transfer...
},
);
Assuming an average call takes 200ms, this optimization reduces the total processing time for 5,000 events from ~16 minutes to ~200 milliseconds - making it 5,000 times faster!
Learn more about the Effect API in our dedicated guide.
Preload Phase Behaviorโ
The Preload Phase is a special phase that runs before the actual event processing. It's designed to preload data that will be used during event processing.
Key characteristics of the Preload Phase:
- It runs in parallel for all events in the batch
- Exceptions won't crash the indexer but will silently abort the Preload Phase for that specific event
- All storage updates are ignored
- All
context.logcalls are ignored
During the second run (Processing Phase), all operations become fully enabled:
- Exceptions will crash the indexer if not handled
- Entity setting operations will persist to the database
- Logging will output to the console
This two-phase design allows the Preload Phase to optimistically attempt loading data that may not exist yet, while ensuring data consistency during the Processing Phase when all operations are executed normally.
If you're using an earlier version of envio, we strongly recommend upgrading to the latest version using pnpm install envio@latest to benefit from this improved Preload Phase behavior.
Double-Run Footgunโ
As mentioned above, the Preload Phase gives a lot of benefits for the event processing, but also it means that you must be aware of its table run nature:
- Never call
fetchor other external calls directly in the handler.- Use the Effect API instead.
- Or use
context.isPreloadto guarantee that the code will run once.
Due to the optimistic nature of the Preload Phase, the Effect API may occasionally execute with stale data, leading to redundant external calls. If you need to ensure that external calls are made with the most up-to-date data, you can use the context.isPreload check to restrict execution to only the processing phase.
Note: This will disable the Preload Optimization for the external calls.
indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event, context }) => {
const sender = await context.Account.get(event.params.from);
if (context.isPreload) {
return;
}
const metadata = await fetch(
`https://api.example.com/metadata/${sender.metadataId}`
);
},
);
Best Practicesโ
- Use
Promise.allto load multiple entities concurrently for better performance - Place database reads and external calls at the beginning of your handler to maximize the benefits of Preload Optimization
- Consider using
context.isPreloadto exit early from the Preload Phase after loading required data
Migrating from Loadersโ
The Preload Optimization for handlers was born from a concept we had before called Loaders. The handlerWithLoader API has been removed in V3 โ move the loader code into the handler and rely on the always-on Preload Phase.
// V2 โ removed in V3
ERC20.Transfer.handlerWithLoader({
loader: async ({ event, context }) => {
// Load sender and receiver accounts efficiently
const sender = await context.Account.get(event.params.from);
const receiver = await context.Account.get(event.params.to);
// Return the loaded data to the handler
return {
sender,
receiver,
};
},
handler: async ({ event, context, loaderReturn }) => {
const { sender, receiver } = loaderReturn;
// Process the transfer with the pre-loaded data
// No database lookups needed here!
},
});
// V3
indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event, context }) => {
// Load sender and receiver accounts efficiently
const sender = await context.Account.get(event.params.from);
const receiver = await context.Account.get(event.params.to);
// To imitate the behavior of the loader,
// we can use `context.isPreload` to make next code run only once.
// Note: This is not required, but might be useful for CPU-intensive operations.
if (context.isPreload) {
return;
}
// Process the transfer with the pre-loaded data
},
);
Effect Apiโ
File: Advanced/effect-api.md
The Effect API provides a powerful and convenient way to perform external calls from your handlers. It's especially effective when used with Preload Optimization:
- Automatic batching: Calls of the same kind are automatically batched together
- Intelligent memoization: Calls are memoized, so you don't need to worry about the handler function being called multiple times
- Deduplication: Calls with the same arguments are deduplicated to prevent overfetching
- Persistence: Built-in support for result persistence for indexer reruns (opt-in via
cache: true) - Future enhancements: We're working on automatic retry logic and enhanced caching workflows ๐๏ธ
To use the Effect API, you first need to define an effect using createEffect function from the envio package:
export const getMetadata = createEffect(
{
name: "getMetadata",
input: S.string,
output: {
description: S.string,
value: S.bigint,
},
rateLimit: {
calls: 5,
per: "second",
},
cache: true,
},
async ({ input, context }) => {
const response = await fetch(`https://api.example.com/metadata/${input}`);
const data = await response.json();
context.log.info(`Fetched metadata for ${input}`);
return {
description: data.description,
value: data.value,
};
}
);
The first argument is an options object that describes the effect:
name(required) - the name of the effect used for debugging and logginginput(required) - the input type of the effectoutput(required) - the output type of the effectrateLimit(required) - the maximum calls allowed per timeframe, orfalseto disablecache(optional) - save effect results in the database to prevent duplicate calls
The second argument is a function that will be called with the effect's input.
Note: For type definitions, you should use
Sfrom theenviopackage, which uses Sury library under the hood.
After defining an effect, you can use context.effect to call it from your handler or another effect.
The context.effect function accepts an effect as the first argument and the effect's input as the second argument:
indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event, context }) => {
const metadata = await context.effect(getMetadata, event.params.from);
// Process the event with the metadata
},
);
Reading On-Chain State (eth_call)โ
The Effect API is how you perform eth_call-style reads from your handlers โ for example, reading a token balance, fetching a contract's name, or querying any view function at a specific block.
Viem Transport Batchingโ
You can use viem or any other blockchain client inside your effect functions. When doing so, it's highly recommended to enable the batch option to group all effect calls into fewer RPC requests:
// Create a public client to interact with the blockchain
const client = createPublicClient({
chain: mainnet,
// Enable batching to group calls into fewer RPC requests
transport: http(rpcUrl, { batch: true }),
});
// Get the contract instance for your contract
const lbtcContract = getContract({
abi: erc20Abi,
address: "0x8236a87084f8B84306f72007F36F2618A5634494",
client: client,
});
// Effect to get the balance of a specific address at a specific block
export const getBalance = createEffect(
{
name: "getBalance",
input: {
address: S.string,
blockNumber: S.optional(S.bigint),
},
output: S.bigint,
rateLimit: {
calls: 5,
per: "second",
},
cache: true,
},
async ({ input, context }) => {
try {
// If blockNumber is provided, use it to get balance at that specific block
const options = input.blockNumber
? { blockNumber: input.blockNumber }
: undefined;
const balance = await lbtcContract.read.balanceOf(
[input.address as `0x${string}`],
options
);
return balance;
} catch (error) {
context.log.error(`Error getting balance for ${input.address}: ${error}`);
// Return 0 on error to prevent processing failures
return BigInt(0);
}
}
);
Persistenceโ
By default, effect results are not persisted in the database. This means if the effect with the same input is called again, the function will be executed the second time.
To persist effect results, you can set the cache option to true when creating the effect. This will save the effect results in the database and reuse them in future indexer runs. You can also override caching for a specific call by setting context.cache = false, which prevents storing results for that execution, especially useful when handling failed responses.
Example setting cache to false with context.cache:
export const getBalance = createEffect(
{
// effect options
cache: true,
},
async ({ input, context }) => {
try {
// your effect logic
} catch (_) {
// Don't cache failed response
context.cache = false;
return undefined;
}
}
);
Every effect cache creates a new table in the database envio_effect_${effectName}. You can see it and query in Hasura console with admin secret.
Also, use our Development Console to track the cache size and see number of calls which didn't hit the cache.
Reuse Effect Cache on Indexer Rerunsโ
To prevent invalid data we don't keep the effect cache on indexer reruns. But you can explicitly configure cache, which should be preloaded when the indexer is rerun.
Open Development Console of the running indexer which accumulated the cache. You'll be able to see the Sync Cache button right at the Effects section. Clicking the button will load the cache from the indexer database to the .envio/cache directory in your indexer project.
When the indexer is rerun by using envio dev or envio start -r call, the initial cache will be loaded from the .envio/cache directory and used for the indexer run.
Note: This feature is available starting from
envio@2.26.0. It also doesn't support rollbacks on reorgs. The support for reorgs will be added in the future.
Cache on Envio Cloudโ
Envio Cloud provides built-in cache management for Effect API results, allowing you to save and restore caches directly from the dashboard without committing files to your repository.
Key Features:
- Save Cache: Capture effect data from any deployment with one click via Quick Actions
- Cache Settings: Manage caches in Settings > Cache - enable/disable caching and select which cache to use
- Automatic Restore: New deployments automatically preload effect data from your selected cache
This eliminates the need to commit .envio/cache to your repository and removes file size limitations.
For detailed instructions, see the Effect API Cache documentation.
Rate Limitโ
Starting from v2.32.0, the rateLimit option was added. It controls how frequently an effect can run within a given timeframe. You can set it to false to disable rate limiting or define a custom limit such as calls per second, minute, or a duration in milliseconds.
// Effect to get the balance of a specific address at a specific block
export const getBalance = createEffect(
{
name: "getBalance",
input: {
address: S.string,
blockNumber: S.optional(S.bigint),
},
output: S.bigint,
// rateLimit: false, // you can set rateLimit to false if needed
rateLimit: {
calls: 5,
per: "second", // also supports "minute" or a duration in milliseconds
},
cache: true,
},
async ({ input, context }) => {
// your effect logic
}
);
Watch the following video to learn more about createEffect and other updates introduced in v2.32.0.
Sending Notifications (Webhooks)โ
You can use the Effect API to send push notifications or webhook calls when specific events occur. This is useful for alerting systems, Discord/Slack bots, or triggering downstream workflows.
export const sendWebhook = createEffect(
{
name: "sendWebhook",
input: {
event: S.string,
data: S.string,
},
output: S.boolean,
rateLimit: {
calls: 10,
per: "second",
},
// Don't cache webhook calls - we want them to fire every time
cache: false,
},
async ({ input, context }) => {
try {
await fetch("https://your-webhook-url.com/notify", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ event: input.event, data: input.data }),
});
return true;
} catch (error) {
context.log.error(`Webhook failed: ${error}`);
return false;
}
}
);
Then call it from your handler:
indexer.onEvent(
{ contract: "MyContract", event: "LargeTransfer" },
async ({ event, context }) => {
await context.effect(sendWebhook, {
event: "large_transfer",
data: JSON.stringify({
from: event.params.from,
to: event.params.to,
amount: event.params.value.toString(),
}),
});
},
);
Webhook effects will fire on every indexer re-run unless you set cache: true. If you cache them, the webhook will only fire once per unique input. Consider which behavior is appropriate for your use case.
Migrate from Experimentalโ
If you're migrating from experimental_createEffect to createEffect, remove the experimental_ prefix and add the rateLimit option, which is now required. In experimental_createEffect, the rateLimit option was optional and defaulted to false.
- export const getBalance = experimental_createEffect(
+ export const getBalance = createEffect(
{
name: "getBalance",
input: {
address: S.string,
blockNumber: S.optional(S.bigint),
},
output: S.bigint,
+ rateLimit: {
+ calls: 5,
+ per: "second",
+ },
cache: true,
},
async ({ input, context }) => {
// your effect logic
}
);
Accessing Contract State in Event Handlersโ
File: Guides/contract-state.md
Example Repository: The complete code for this guide can be found here
Introductionโ
This guide demonstrates how to access on-chain contract state from your event handlers. You'll learn how to:
- Make RPC calls to external contracts within your event handlers
- Batch multiple calls using multicall for efficiency
- Learn about Preload Optimisation and how it makes your indexer thousands of times faster
- Use Effect API with built-in caching and Viem transport level batching
- Handle common edge cases that arise when accessing token contract data
The Challenge: Token Data from Pool Creation Eventsโ
Scenarioโ
We want to track token information (name, symbol, decimals) for every token involved in a Uniswap V3 pool creation event.
Problemโ
The Uniswap V3 factory PoolCreated event only provides token addresses, not their metadata:
PoolCreated(address indexed token0, address indexed token1, uint24 indexed fee, int24 tickSpacing, address pool)
To get the token name, symbol, and decimals, we need to:
- Extract the token addresses from the event
- Make RPC calls to each token's contract
- Store this data alongside our pool information
Prerequisitesโ
This guide assumes:
- Basic familiarity with Envio indexing
- Understanding of the viem library for making contract calls
- Access to an Ethereum RPC endpoint (dRPC recommended)
For a gentle introduction to viem with a similar example, check out this medium article.
Implementation Stepsโ
Step 1: Setup the Indexer Configurationโ
First, create a new indexer:
pnpx envio init
When prompted, enter the Ethereum mainnet Uniswap V3 Factory address: 0x1F98431c8aD98523631AE4a59f267346ea31F984
Then modify your configuration to focus only on the PoolCreated event:
# config.yaml
name: uniswap-v3-factory-token-indexer
chains:
- id: 1
start_block: 0
contracts:
- name: UniswapV3Factory
address:
- "0x1F98431c8aD98523631AE4a59f267346ea31F984"
events:
- event: PoolCreated(address indexed token0, address indexed token1, uint24 indexed fee, int24 tickSpacing, address pool)
Step 2: Define the Schemaโ
Create a schema that captures both pool and token information:
# schema.graphql
type Token {
id: ID! # token address
name: String!
symbol: String!
decimals: Int!
}
type Pool {
id: ID! # unique identifier
token0: Token!
token1: Token!
fee: BigInt!
tickSpacing: BigInt!
pool: String! # pool address
}
Step 3: Implement the Event Handlerโ
The event handler needs to:
- Create a Pool entity from the event data
- Make RPC calls to fetch token information for both token0 and token1
- Create Token entities with the retrieved data
Important! Preload optimization makes your handlers run twice. So instead of direct RPC calls, we're doing it through context.effect - the Effect API.
Learn how Preload Optimization works in a dedicated guide. It might be a new mental model for you, but this is what can make indexing thousands of times faster.
// src/handlers
indexer.onEvent(
{ contract: "UniswapV3Factory", event: "PoolCreated" },
async ({ event, context }) => {
// Create Pool entity
context.Pool.set({
id: `${event.chainId}_${event.block.number}_${event.logIndex}`,
token0_id: event.params.token0,
token1_id: event.params.token1,
fee: event.params.fee,
tickSpacing: event.params.tickSpacing,
pool: event.params.pool,
});
// Fetch and store token0 information
try {
const tokenMetadata0 = await context.effect(getTokenMetadata, {
tokenAddress: event.params.token0,
chainId: event.chainId,
});
context.Token.set({
id: event.params.token0,
name: tokenMetadata0.name,
symbol: tokenMetadata0.symbol,
decimals: tokenMetadata0.decimals,
});
} catch (error) {
context.log.error("Failed to fetch token0 metadata", {
tokenAddress: event.params.token0,
chainId: event.chainId,
pool: event.params.pool,
err: error,
});
return;
}
// Fetch and store token1 information
try {
const tokenMetadata1 = await context.effect(getTokenMetadata, {
tokenAddress: event.params.token1,
chainId: event.chainId,
});
context.Token.set({
id: event.params.token1,
name: tokenMetadata1.name,
symbol: tokenMetadata1.symbol,
decimals: tokenMetadata1.decimals,
});
} catch (error) {
context.log.error("Failed to fetch token1 metadata", {
tokenAddress: event.params.token1,
chainId: event.chainId,
pool: event.params.pool,
err: error,
});
return;
}
},
);
Step 4: Create the Token Metadata Effectโ
This is where the magic happens. We need to:
- Make RPC calls to token contracts
- Use multicall to batch multiple calls for efficiency
- Handle edge cases like non-standard ERC20 implementations
- Cache results to avoid redundant calls
// src/tokenDetails.ts
const RPC_URL = process.env.RPC_URL;
const client = createPublicClient({
chain: mainnet,
batch: { multicall: true }, // Enable multicall batching for efficiency
transport: http(RPC_URL, { batch: true }), // Thanks to automatic Effect API batching, we can also enable batching for Viem transport level
});
// Use Sury library to define the schema
const tokenMetadataSchema = S.schema({
name: S.string,
symbol: S.string,
decimals: S.number,
});
// Infer the type from the schema
type TokenMetadata = S.Infer;
export const getTokenMetadata = createEffect(
{
name: "getTokenMetadata",
input: {
tokenAddress: S.string,
chainId: S.number,
},
output: tokenMetadataSchema,
rateLimit: {
calls: 5,
per: "second",
},
// Enable caching to avoid duplicated calls
cache: true,
},
async ({ input, context }) => {
const { tokenAddress, chainId } = input;
// Prepare contract instances for different token standard variations
const erc20 = getERC20Contract(tokenAddress as `0x${string}`);
const erc20Bytes = getERC20BytesContract(tokenAddress as `0x${string}`);
let results: [number, string, string];
try {
// Try standard ERC20 interface first (most common)
results = await client.multicall({
allowFailure: false,
contracts: [
{
...erc20,
functionName: "decimals",
},
{
...erc20,
functionName: "name",
},
{
...erc20,
functionName: "symbol",
},
],
});
} catch (error) {
try {
// Some tokens use bytes32 for name/symbol instead of string
const alternateResults = await client.multicall({
allowFailure: false,
contracts: [
{
...erc20Bytes,
functionName: "decimals",
},
{
...erc20Bytes,
functionName: "name",
},
{
...erc20Bytes,
functionName: "symbol",
},
],
});
results = [
alternateResults[0],
hexToString(alternateResults[1]).replace(/\u0000/g, ""), // Remove null byte padding
hexToString(alternateResults[2]).replace(/\u0000/g, ""), // Remove null byte padding
];
} catch (alternateError) {
results = [0, "unknown", "unknown"]; // Fallback for completely non-standard tokens
}
}
const [decimals, name, symbol] = results;
return {
name,
symbol,
decimals,
};
}
);
Important: The
hexToStringmethod from Viem adds byte padding to the string. We remove this padding withreplace(/\u0000/g, '')to avoid errors when writing to the database.
Note: Read more about Effect API and caching in the Effect API guide.
Key Considerationsโ
Understanding Current vs. Historical Stateโ
Standard RPC requests return the current state of a contract, not the state at a specific historical block. For token metadata (name, symbol, decimals), this isn't typically an issue since these values rarely change.
However, if you need historical state (like an account balance at a specific block), you would need a specialized RPC method like eth_getBalanceAt.
Handling Rate Limitingโ
RPC providers often limit the number of requests per time period. To avoid hitting rate limits:
- Use multicall (as shown in our example) to batch multiple contract calls into a single RPC request
- Learn about Preload Optimization to make your indexer thousands of times faster
- Enable caching to avoid redundant requests
- Use a paid, unthrottled RPC provider for production indexers
- Implement request throttling to space out requests when needed
- Use multiple RPC providers and rotate between them for high-volume indexing
Conclusionโ
Accessing contract state from your event handlers opens up powerful possibilities for enriching your indexed data. By following the patterns in this guide, you can efficiently retrieve and store contract state while maintaining good performance.
For more advanced techniques, explore:
- Implementing retry logic for failed RPC calls
- Handling complex contract interactions beyond basic ERC20 tokens
Indexing IPFS Data with Envioโ
File: Guides/ipfs.md
Example Repository: The complete code for this guide can be found here
Introductionโ
This guide demonstrates how to fetch and index data stored on IPFS within your Envio indexer. We'll use the Bored Ape Yacht Club NFT collection as a practical example, showing you how to retrieve and store token metadata from IPFS.
IPFS (InterPlanetary File System) is commonly used in blockchain applications to store larger data like images and metadata that would be prohibitively expensive to store on-chain. By integrating IPFS fetching capabilities into your indexers, you can provide a more complete data model that combines on-chain events with off-chain metadata.
Implementation Overviewโ
Our implementation will follow these steps:
- Create a basic indexer for Bored Ape Yacht Club NFT transfers
- Extend the indexer to fetch and store metadata from IPFS
- Handle IPFS connection issues with fallback gateways
Step 1: Setting Up the Basic NFT Indexerโ
First, let's create a basic indexer that tracks NFT ownership:
Initialize the Indexerโ
pnpx envio init
When prompted, enter the Bored Ape Yacht Club contract address: 0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D
Configure the Indexerโ
Modify the configuration to focus on the Transfer events:
# config.yaml
name: bored-ape-yacht-club-nft-indexer
chains:
- id: 1
start_block: 0
end_block: 12299114 # Optional: limit blocks for development
contracts:
- name: BoredApeYachtClub
address:
- "0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D"
events:
- event: Transfer(address indexed from, address indexed to, uint256 indexed tokenId)
Define the Schemaโ
Create a schema to store NFT ownership data:
# schema.graphql
type Nft {
id: ID! # tokenId
owner: String!
}
Implement the Event Handlerโ
Track ownership changes by handling Transfer events:
// src/EventHandler.ts
const ZERO_ADDRESS = "0x0000000000000000000000000000000000000000";
indexer.onEvent(
{ contract: "BoredApeYachtClub", event: "Transfer" },
async ({ event, context }) => {
if (event.params.from === ZERO_ADDRESS) {
// mint
context.Nft.set({
id: event.params.tokenId.toString(),
owner: event.params.to,
});
} else {
// transfer
const nft = await context.Nft.getOrThrow(event.params.tokenId.toString());
context.Nft.set({
...nft,
owner: event.params.to,
});
}
},
);
Run your indexer with pnpm dev and visit http://localhost:8080 to see the ownership data:
!Basic NFT ownership data
Step 2: Fetching IPFS Metadataโ
Now, let's enhance our indexer to fetch metadata from IPFS:
Update the Schemaโ
Extend the schema to include metadata fields:
# schema.graphql
type Nft {
id: ID! # tokenId
owner: String!
image: String!
attributes: String! # JSON string of attributes
}
Create IPFS Effectโ
Important! Preload optimization makes your handlers run twice. So instead of direct RPC calls, we're doing it through the Effect API.
Learn how Preload Optimization works in a dedicated guide. It might be a new mental model for you, but this is what can make indexing thousands of times faster.
Let's create the getIpfsMetadata effect in the src/utils/ipfs.ts file:
// Define the schema for the IPFS metadata
// It uses Sury library to define the schema
const nftMetadataSchema = S.schema({
image: S.string,
attributes: S.string,
});
// Infer the type from the schema
type NftMetadata = S.Infer;
// Unique identifier for the BoredApeYachtClub IPFS tokenURI
const BASE_URI_UID = "QmeSjSinHpPnmXmspMjwiXyN6zS4E9zccariGR3jxcaWtq";
const endpoints = [
// Try multiple endpoints to ensure data availability
// Optional paid gateway (set in .env)
...(process.env.PINATA_IPFS_GATEWAY ? [process.env.PINATA_IPFS_GATEWAY] : []),
"https://cloudflare-ipfs.com/ipfs",
"https://ipfs.io/ipfs",
];
async function fetchFromEndpoint(
context: EffectContext,
endpoint: string,
tokenId: string
): Promise {
try {
const response = await fetch(`${endpoint}/${BASE_URI_UID}/${tokenId}`);
if (response.ok) {
const metadata: any = await response.json();
return {
image: metadata.image,
attributes: JSON.stringify(metadata.attributes),
};
} else {
context.log.warn(`IPFS didn't return 200`, { tokenId, endpoint });
return null;
}
} catch (e) {
context.log.warn(`IPFS fetch failed`, { tokenId, endpoint, err: e });
return null;
}
}
export const getIpfsMetadata = createEffect(
{
name: "getIpfsMetadata",
input: S.string,
output: nftMetadataSchema,
rateLimit: {
calls: 5,
per: "second",
},
},
async ({ input: tokenId, context }) => {
for (const endpoint of endpoints) {
const metadata = await fetchFromEndpoint(context, endpoint, tokenId);
if (metadata) {
return metadata;
}
}
// โ ๏ธ Dangerous: Sometimes it's better to crash, to prevent corrupted data
// But we're going to use a fallback value, to keep the indexer process running.
// Both approaches have their pros and cons.
context.log.warn(
"Unable to fetch IPFS. Continuing with fallback metadata.",
{
tokenId,
}
);
return { attributes: `["unknown"]`, image: "unknown" };
}
);
Update the Event Handlerโ
Let's modify the event handler to fetch and store metadata using the getIpfsMetadata effect:
// src/handlers
const ZERO_ADDRESS = "0x0000000000000000000000000000000000000000";
indexer.onEvent(
{ contract: "BoredApeYachtClub", event: "Transfer" },
async ({ event, context }) => {
if (event.params.from === ZERO_ADDRESS) {
// mint
const metadata = await context.effect(
getIpfsMetadata,
event.params.tokenId.toString()
);
context.Nft.set({
id: event.params.tokenId.toString(),
owner: event.params.to,
image: metadata.image,
attributes: metadata.attributes,
});
} else {
// transfer
const nft = await context.Nft.getOrThrow(event.params.tokenId.toString());
context.Nft.set({
...nft,
owner: event.params.to,
});
}
},
);
When you run the indexer now, it will populate both ownership data and token metadata:
!NFT ownership and metadata
Best Practices for IPFS Integrationโ
When working with IPFS in your indexers, consider these best practices:
1. Use Multiple Gatewaysโ
IPFS gateways can be unreliable, so always implement multiple fallback options:
const endpoints = [
...(process.env.PAID_IPFS_GATEWAY ? [process.env.PAID_IPFS_GATEWAY] : []),
"https://cloudflare-ipfs.com/ipfs",
"https://ipfs.io/ipfs",
"https://gateway.pinata.cloud/ipfs",
];
2. Handle Failures Gracefullyโ
Always include error handling and provide fallback values:
try {
// IPFS fetch logic
} catch (error) {
context.log.error(`Failed to fetch from IPFS`, error as Error);
return { attributes: [], image: "default-image-url" };
}
3. Implement Local Caching (For Local Development)โ
Follow the Effect API Persistence guide to implement caching for local development. This should allow you to avoid repeatedly fetching the same data.
export const getIpfsMetadata = createEffect(
{
name: "getIpfsMetadata",
input: S.string,
output: nftMetadataSchema,
rateLimit: {
calls: 5,
per: "second",
},
cache: true, // Enable caching
},
async ({ input: tokenId, context }) => {...}
);
Important: While the example repository includes SQLite-based caching, this approach is outdated and leads to many indexing issues.
Note: We're working on a better integration with Envio Cloud. Currently, due to the cache size, it's not recommended to commit the
.envio/cachedirectory to the GitHub repository.
4. Learn about Preload Optimizationโ
Learn how Preload Optimization works and the Double-Run Footgun in a dedicated guide. It might be a new mental model for you, but this is what can make indexing thousands of times faster.
Understanding IPFSโ
What is IPFS?โ
IPFS (InterPlanetary File System) is a distributed system for storing and accessing files, websites, applications, and data. It works by:
- Splitting files into chunks
- Creating content-addressed identifiers (CIDs) based on the content itself
- Distributing these chunks across a network of nodes
- Retrieving data based on its CID rather than its location
Common Use Cases with Smart Contractsโ
IPFS is frequently used alongside smart contracts for:
- NFTs: Storing images, videos, and metadata while the contract manages ownership
- Decentralized Identity Systems: Storing credential documents and personal information
- DAOs: Maintaining governance documents, proposals, and organizational assets
- dApps: Hosting front-end interfaces and application assets
IPFS Challengesโ
IPFS integration comes with several challenges:
- Slow Retrieval Times: IPFS data can be slow to retrieve, especially for less widely replicated content
- Gateway Reliability: Public gateways can be inconsistent in their availability
- Data Persistence: Content may become unavailable if nodes stop hosting it
To mitigate these issues:
- Use pinning services like Pinata or Infura to ensure data persistence
- Implement multiple gateway fallbacks
- Consider paid gateways for production applications
Using HyperSync as Your Indexing Data Sourceโ
File: Advanced/hypersync.md
"Beam me up, Scotty!" ๐ โ Just like the Star Trek transporter, HyperSync delivers your blockchain data at warp speed.
What is HyperSync?โ
HyperSync is a purpose built data-node that helps powers the exceptional performance of HyperIndex. It's a specialized data source optimized for indexing that provides:
- 2000x faster sync speeds compared to traditional RPC methods
- Cost-effective data retrieval with optimized resource usage
- Flexibility with the ability to fetch multiple data points in a single round trip with more complex filtering
How HyperSync Powers Your Indexersโ
The Performance Advantageโ
Traditional blockchain indexing relies on RPC (Remote Procedure Call) endpoints to query blockchain data. While functional, RPCs become highly inefficient when:
- Indexing millions of events
- Processing historical blockchain data
- Extracting data across multiple networks
- Working with thousands of contracts
HyperSync addresses these limitations by providing a streamlined data access layer that dramatically reduces sync times from days to minutes.
Default Enablementโ
HyperSync is used by default as the data source for all HyperIndex chains. This means:
- No additional configuration is required to benefit from its speed
- No need to worry about RPC rate limiting
- No management of multiple RPC providers
- No costs for external RPC services
Starting in V3, HyperSync requires an API token. Create a free token at envio.dev/app/api-tokens and expose it to your indexer as ENVIO_API_TOKEN:
export ENVIO_API_TOKEN=your_token_here
Using HyperSync in Your Projectsโ
Configurationโ
To use HyperSync (the default), simply don't set an RPC for historical sync in your config. HyperIndex will automatically use HyperSync for supported chains:
name: Greeter
description: Greeter indexer
chains:
- id: 137 # Polygon
start_block: 0 # With HyperSync, you can use 0 regardless of contract deployment time
contracts:
- name: PolygonGreeter
abi_file_path: abis/greeter-abi.json
address: "0x9D02A17dE4E68545d3a58D3a20BbBE0399E05c9c"
events:
- event: NewGreeting
- event: ClearGreeting
Smart Block Detectionโ
When using HyperSync, you can specify start_block: 0 in your configuration. HyperSync will automatically:
- Detect the first block where your contract was deployed
- Begin indexing from that block
- Skip unnecessary processing of earlier blocks
This feature eliminates the need to manually determine the deployment block of your contract, saving setup time and reducing configuration errors.
Hosting and Supportโ
HyperSync is maintained and hosted by Envio for all supported networks. We handle the infrastructure, allowing you to focus on building your indexer logic.
Supported Chainsโ
HyperSync supports numerous EVM chains including Ethereum, Unichain, Arbitrum, Optimism, and more. For a complete and up-to-date list of supported chains, see the HyperSync Supported Networks documentation.
Alternative Data Sourcesโ
HyperSync data source is vendorlock-free. While HyperSync is recommended for optimal performance, you can always switch to RPCs without the need to change your indexer code. For information on configuring RPC-based indexing, visit the RPC Data Source documentation.
Improving Resilience with RPC fallbackโ
For production deployments, itโs recommended to use HyperSync as the primary data source and have RPCs as fallback to improve reliability.You can read more about it in the RPC Fallback section.
Performance Comparisonโ
| Metric | Traditional RPC | HyperSync |
|---|---|---|
| Indexing 1M Events | Hours to days | Minutes |
| Resource Usage | High | Optimized |
| Network Calls | Many individual calls | Batched for efficiency |
| Rate Limiting | Common issue | Not applicable |
| Cost | Pay per API call | Included with Envio Cloud |
Summaryโ
HyperSync provides a significant competitive advantage for Envio indexers by dramatically reducing sync times, lowering costs, and simplifying configuration. By using HyperSync as your default data source, you'll experience:
- Faster indexing performance
- Support for previously impossible indexing cases
- Enhanced reliability
- Reduced operational complexity
To learn more about HyperSync's underlying technology, visit the HyperSync documentation.
Using RPC as Your Indexing Data Sourceโ
File: Advanced/rpc-sync.md
HyperIndex supports indexing any EVM blockchain using RPC (Remote Procedure Call) as the data source. This page explains when and how to use RPC for your indexing needs.
When to Use RPCโ
While HyperSync is the recommended and default data source for optimal performance, there are scenarios where you might need to use RPC instead:
- Unsupported Chains: When indexing a blockchain that isn't yet supported by HyperSync
- Custom Requirements: When you need specific RPC functionality not available in HyperSync
- Private Chains: When working with private or development EVM chains
Note: For chains that HyperSync supports, we strongly recommend using HyperSync rather than RPC. HyperSync provides significantly faster indexing performance (up to 100x) and doesn't require managing RPC endpoints or worrying about rate limits.
Configuring RPC in Your Indexerโ
Basic Configurationโ
In V3 the V2 rpc_config field has been replaced with rpc, which accepts a single URL, a single Rpc object, or a list of Rpc objects. Each entry can declare what it's for: sync (historical), realtime (head, including WebSocket URLs), or fallback.
To use RPC as the primary historical data source, add an rpc entry with for: sync to your chain configuration in config.yaml:
chains:
- id: 1 # Ethereum Mainnet
rpc:
- url: https://eth-mainnet.your-rpc-provider.com # Your RPC endpoint
for: sync
start_block: 15000000
contracts:
- name: MyContract
address: "0x1234..."
# Additional contract configuration...
The presence of an RPC marked for: sync tells HyperIndex to use RPC instead of HyperSync for historical sync on this chain. You can also add a for: realtime WebSocket endpoint to follow the head:
chains:
- id: 1
rpc:
- url: https://eth-mainnet.your-rpc-provider.com
for: sync
- url: wss://eth-mainnet.your-rpc-provider.com
for: realtime
Advanced RPC Configurationโ
For more control over how your indexer interacts with the RPC endpoint, you can configure additional parameters:
chains:
- id: 1
rpc:
- url: https://eth-mainnet.your-rpc-provider.com
for: sync
initial_block_interval: 10000 # Initial number of blocks to fetch in each request
backoff_multiplicative: 0.8 # Factor to scale back block request size after errors
acceleration_additive: 2000 # How many more blocks to request when successful
interval_ceiling: 10000 # Maximum blocks to request in a single call
backoff_millis: 5000 # Milliseconds to wait after an error
query_timeout_millis: 20000 # Milliseconds before timing out a request
start_block: 15000000
# Additional chain configuration...
Configuration Parameters Explainedโ
| Parameter | Description | Recommended Value |
|---|---|---|
url | Your RPC endpoint URL | Depends on provider |
initial_block_interval | Starting block batch size | 1,000 - 10,000 |
backoff_multiplicative | How much to reduce batch size after errors | 0.5 - 0.9 |
acceleration_additive | How much to increase batch size on success | 500 - 2,000 |
interval_ceiling | Maximum blocks per request | 5,000 - 10,000 |
backoff_millis | Wait time after errors (ms) | 1,000 - 10,000 |
query_timeout_millis | Request timeout (ms) | 10,000 - 30,000 |
The optimal values depend on your RPC provider's performance and limits, as well as the complexity of your contracts and the data being indexed.
RPC Best Practicesโ
Selecting an RPC Providerโ
When choosing an RPC provider, consider:
- Rate limits: Most providers have limits on requests per second/minute
- Node performance: Some providers offer faster nodes for premium tiers
- Archive nodes: Required if you need historical state (e.g., balances at past blocks)
- Geographic location: Choose nodes closest to your indexer deployment
Performance Optimizationโ
To get the best performance when using RPC:
- Start from a recent block if possible, rather than indexing from genesis
- Tune batch parameters based on your provider's capabilities
- Use a paid service for better reliability and higher rate limits
- Consider multiple fallback RPCs for redundancy
Improving resilience with RPC fallbackโ
HyperIndex allows you to configure additional RPC providers as fallback data sources. This redundancy is recommended for production deployments to ensure continuous operation of your indexer. If HyperSync experiences any interruption, your indexer will automatically switch to the fallback RPC provider.
Adding an RPC fallback provides these benefits:
- High availability: Your indexer continues to function even during temporary HyperSync outages
- Automatic failover: The system detects issues and switches to fallback RPC without manual intervention
- Operational control: You can specify which RPC providers to use as fallbacks based on your requirements
Configure a fallback RPC by adding the rpc field to your chain configuration:
name: Greeter
description: Greeter indexer
chains:
- id: 137 # Polygon
+ # Short and simple
+ rpc: https://polygon.your-rpc-provider.com?API_KEY={ENVIO_POLYGON_API_KEY}
+ # Or provide multiple RPC endpoints with more flexibility
+ rpc:
+ - url: https://polygon.your-rpc-provider.com?API_KEY={ENVIO_POLYGON_API_KEY}
+ for: fallback
+ - url: https://polygon.your-free-rpc-provider.com
+ for: fallback
+ initial_block_interval: 1000
start_block: 0 # With HyperSync, you can use 0 regardless of contract deployment time
contracts:
- name: PolygonGreeter
abi_file_path: abis/greeter-abi.json
address: 0x9D02A17dE4E68545d3a58D3a20BbBE0399E05c9c
events:
- event: NewGreeting
- event: ClearGreeting
The fallback RPC is activated only when a primary data source doesn't receive a new block for more than 20 seconds.
Enhanced RPC with eRPCโ
For more robust RPC usage, you can implement eRPC - a fault-tolerant EVM RPC proxy with advanced features like caching and failover.
What eRPC Providesโ
- Permanent caching: Stores historical responses to reduce redundant requests
- Auto failover: Automatically switches between multiple RPC providers
- Re-org awareness: Properly handles blockchain reorganizations
- Auto-batching: Optimizes requests to minimize network overhead
- Load balancing: Distributes requests across multiple providers
Setting Up eRPCโ
- Create your eRPC configuration file (
erpc.yaml):
logLevel: debug
projects:
- id: main
upstreams:
# Add HyperRPC as primary source
- endpoint: evm+envio://rpc.hypersync.xyz
# Add fallback RPC endpoints
- endpoint: https://eth-mainnet-provider1.com
- endpoint: https://eth-mainnet-provider2.com
- endpoint: https://eth-mainnet-provider3.com
- Run eRPC using Docker:
docker run -v $(pwd)/erpc.yaml:/root/erpc.yaml -p 4000:4000 -p 4001:4001 ghcr.io/erpc/erpc:latest
Or add it to your existing Docker Compose setup:
services:
# Your existing services...
erpc:
image: ghcr.io/erpc/erpc:latest
platform: linux/amd64
volumes:
- "${PWD}/erpc.yaml:/root/erpc.yaml"
ports:
- 4000:4000
- 4001:4001
restart: always
- Configure HyperIndex to use eRPC in your
config.yaml:
chains:
- id: 1
rpc:
- url: http://erpc:4000/main/evm/1 # eRPC endpoint for Ethereum Mainnet
for: sync
start_block: 15000000
# Additional chain configuration...
For more detailed configuration options, refer to the eRPC documentation.
Comparing HyperSync and RPCโ
| Feature | HyperSync | RPC |
|---|---|---|
| Speed | 10-100x faster | Baseline |
| Configuration | Minimal | Requires tuning |
| Rate Limits | None | Depends on provider |
| Cost | Included with Envio Cloud | Pay per request/subscription |
| Chain Support | Supported chains | Any EVM chain |
| Maintenance | Managed by Envio | Self-managed |
Summaryโ
While RPC provides the flexibility to index any EVM blockchain, it comes with performance limitations and configuration complexity. For supported networks, we recommend using HyperSync as your data source for optimal performance.
If you must use RPC:
- Choose a reliable provider
- Configure your indexer for optimal performance
- Consider implementing eRPC for enhanced reliability and performance
- Start from recent blocks when possible to reduce indexing time
For any questions about using RPC with HyperIndex, please contact the Envio team.
Config Schema Referenceโ
File: Advanced/config-schema-reference.md
Static, deep-linkable reference for the V3 config.yaml schema.
Tip: Use the Table of Contents to jump to a field or definition.
Top-level Propertiesโ
- description
- name (required)
- ecosystem
- schema
- contracts
- chains (required)
- rollback_on_reorg
- save_full_history
- field_selection
- raw_events
- address_format
- full_batch_size
- storage
descriptionโ
Description of the project
- type:
string | null
Example (config.yaml):
description: Greeter indexer
nameโ
Name of the project
- type:
string
Example (config.yaml):
name: MyIndexer
ecosystemโ
Ecosystem of the project.
- type:
anyOf(object | null)
Variants:
1: EcosystemTag2:null
Example (config.yaml):
ecosystem: evm
schemaโ
Custom path to schema.graphql file
- type:
string | null
Example (config.yaml):
schema: ./schema.graphql
contractsโ
Global contract definitions that must contain all definitions except addresses. You can share a single handler/abi/event definitions for contracts across multiple chains.
- type:
array | null
Example (config.yaml):
contracts:
- name: Greeter
events:
- event: "NewGreeting(address user, string greeting)"
chainsโ
Configuration of the blockchain chains that the project is deployed on.
- type:
array> - items:
object - items ref: Chain
Example (config.yaml):
chains:
- id: 1
start_block: 0
contracts:
- name: Greeter
address: "0x9D02A17dE4E68545d3a58D3a20BbBE0399E05c9c"