> For the complete documentation index, see [llms.txt](https://docs.envio.dev/llms.txt).

# Benchmarking Your Indexer

Every HyperIndex indexer reports what it's doing over an HTTP metrics endpoint
while it runs. There's no benchmarking mode to enable and no profiler to attach
- if the indexer is running, the numbers are already there.

This page turns those numbers into an answer to one question: **what is my
indexer waiting on, and what do I do about it?**

Work through it in order:

1. [Measure throughput](#step-1-measure-throughput) - how fast is it, in events per second?
2. [Find the bottleneck](#step-2-find-the-bottleneck) - which of five things is it waiting on?
3. [Fix it](#step-3-fix-the-bottleneck) - the section for your bottleneck says what to change.
4. [Prove the fix worked](#step-4-prove-the-fix-worked) - re-measure and compare.

## Step 1: Measure throughput

**Start the indexer** in one terminal - locally with
[`envio dev`](/docs/HyperIndex/running-locally), or in production with
`envio start`:

```bash
pnpm envio dev
```

**Read its metrics** from a second terminal, in the same project:

```bash
pnpm envio metrics          # indexing metrics
pnpm envio metrics runtime  # Node.js process metrics (CPU, memory, GC, event-loop lag)
```

The output is plain Prometheus text - a `# HELP` line describing each metric, a
`# TYPE` line, then one sample per line with its labels in braces:

```text
# HELP envio_progress_events The number of events processed and reflected in the database.
# TYPE envio_progress_events gauge
envio_progress_events{chainId="1"} 158205

# HELP envio_process_elapsed_seconds How long the indexer has been running.
# TYPE envio_process_elapsed_seconds gauge
envio_process_elapsed_seconds 45.801
```

Because every metric documents itself, grepping for the metric you care about -
`pnpm envio metrics | grep envio_progress` - is usually the fastest way to
answer a specific question.

**Take two snapshots** a minute or so apart, and divide the change in events by
the change in elapsed time:

```bash
pnpm envio metrics > t1.txt
# wait ~60s
pnpm envio metrics > t2.txt

grep -E '^envio_(progress_events|process_elapsed_seconds)' t1.txt t2.txt
# t1: 158205 events at 45.8s
# t2: 262000 events at 105.8s
# (262000 - 158205) / (105.8 - 45.8) = ~1730 events/second
```

`envio_progress_events` is reported per `chainId`, so a multichain indexer prints
one sample per chain: compare the same `chainId` across both snapshots for a
per-chain figure, or sum every chain within each snapshot for the indexer as a
whole. Both snapshots also have to come from one uninterrupted run - restarting
the indexer resets `envio_process_elapsed_seconds`, so start a fresh baseline
after any restart. (Prometheus `rate()` handles that for you when you scrape
instead.)

Interpret the result against what your workload plausibly allows:

| Events per second | Reading |
|-------------------|---------|
| Over 10,000 | Excellent - most likely bounded by the data source, not by you |
| 1,000–5,000 | Good. Worth tuning only if your sync time is still too long |
| Under 500 | Something is probably wrong. Continue to step 2 |

:::tip Measure over historical sync, not at the head
Once a chain is caught up (`envio_progress_ready` is `1`), throughput reflects
how fast new blocks arrive, not how fast your indexer is. Benchmark while the
chain is still syncing history, or against a fixed block range with an
`end_block` in [`config.yaml`](/docs/HyperIndex/configuration-file), so two runs
are comparable.
:::

## Step 2: Find the bottleneck

A slow indexer is waiting on something. Two counters attribute that wait
directly - read them first, because they point at the answer instead of hinting
at it:

```bash
pnpm envio metrics | grep -E 'stalled_on|process_elapsed'
```

| Metric | Meaning | Go to |
|--------|---------|-------|
| `envio_processing_stalled_on_fetch_seconds` | Nothing to process - events hadn't been fetched yet | [Fetching](#fetching) |
| `envio_processing_stalled_on_storage_write_seconds` | Processing paused - too many changes queued for writing | [Storage writes](#storage-writes) |

Both are cumulative seconds, so read them as a share of the run:

```promql
envio_processing_stalled_on_fetch_seconds / envio_process_elapsed_seconds
```

Anything above roughly `0.3` (30% of the run) is worth acting on. The same
division works for every `_seconds` counter - that's how you compare parts of
the pipeline against each other:

| Metric | Share of the run spent… |
|--------|-------------------------|
| `envio_processing_seconds` | inside your [event handlers](#handlers) |
| `envio_preload_seconds` | [loading entities](#entity-loading) for a batch |
| `envio_storage_write_seconds` | [writing batches](#storage-writes) to storage |
| `envio_fetching_block_range_seconds` | [fetching](#fetching) block ranges (per `chainId`) |
| `envio_effect_call_seconds_total` | inside [Effect API](#external-calls) calls (per `effect`) |

:::note Counters, counts, and gauges
Only `_seconds` counters are durations you can divide by elapsed time. Counts
(`envio_progress_events`, `envio_processing_handler_total`) and gauges
(`envio_progress_block`, `envio_indexing_concurrency`) are read as values or as
deltas between snapshots. `envio_process_metric_time_seconds` records when a
snapshot was taken, so you can compute rates between two of them by hand.
:::

If you scrape with Prometheus instead of reading snapshots, every counter above
works as a rate - `rate(envio_processing_stalled_on_fetch_seconds[5m])` - see
[Scraping with Prometheus](/docs/HyperIndex/observability#scraping-with-prometheus).

## Step 3: Fix the bottleneck

Each section below covers one bottleneck: how you recognise it, what it means,
and what to change. Work only the one that step 2 pointed you at.

### Fetching

**How you know**

```bash
pnpm envio metrics | grep -E 'stalled_on_fetch|indexing_idle|fetching_block_range_seconds'
```

`envio_processing_stalled_on_fetch_seconds` is a large share of the run, and
`envio_fetching_block_range_seconds` dominates the other `_seconds` counters.
`envio_indexing_idle_seconds` climbing on a specific `chainId` tells you which
chain is holding you up.

**What it means**

Your handlers are idle waiting for data. Time spent waiting at the chain head
for new blocks is deliberately excluded from the stall counter, so a fully
synced indexer never looks stalled here.

**What to do**

- Use [HyperSync](/docs/HyperIndex/hypersync) if it supports your chain - it's the single biggest change available, and RPC is orders of magnitude slower.
- On [RPC](/docs/HyperIndex/rpc-sync), a faster provider or a higher rate limit is the fix; check `envio_source_request_seconds_total` per `source` to see what each one costs you.
- Compare `envio_indexing_concurrency` with `envio_indexing_partitions` for the chain - if concurrency sits at its ceiling while events still arrive slowly, the source is saturated, not the indexer.
- Watch `envio_fetching_block_range_parse_seconds`: when parsing rivals fetching, you're decoding far more events than your handlers use, so narrow the events in [`config.yaml`](/docs/HyperIndex/configuration-file).

### Entity loading

**How you know**

```bash
pnpm envio metrics | grep -E 'preload_seconds|storage_load'
```

`envio_preload_seconds` or `envio_storage_load_seconds_total` is a large share of
the run. `envio_storage_load_size` and `envio_storage_load_where_size` (labeled
by `operation` and `storage`) show which loads pull the most data.

**What it means**

[Preload optimization](/docs/HyperIndex/preload-optimization) batches the entity
reads of a whole event batch into a few queries - but only for reads it can see
during the preload phase. A read hidden behind a condition that only some events
reach, or one that depends on the result of an earlier read, falls back to a
single-event round trip.

**What to do**

- Read entities unconditionally near the top of the handler so the preload phase can batch them, rather than inside a branch.
- Avoid chaining reads (`get` → use the id you just read → `get` again); load both entities up front where possible.
- Add [database indices](/docs/HyperIndex/database-performance-optimization) for the fields you filter on with `getWhere`.
- Reduce how much each load returns: a `getWhere` that matches thousands of rows costs more than the handler usually needs.

### Handlers

**How you know**

```bash
pnpm envio metrics | grep -E 'processing_handler_(seconds|total)'
```

`envio_processing_seconds` is a large share of the run. To find the specific
handler, divide cumulative time by call count - both are labeled by `contract`
and `event`:

```promql
envio_processing_handler_seconds / envio_processing_handler_total
```

**What it means**

Your own code is the cost. A mean above ~1ms per call is worth a look; compare
handlers against each other rather than against an absolute number.

**What to do**

- Move external calls (RPC, HTTP, IPFS) out of the handler and into the [Effect API](/docs/HyperIndex/effect-api), which batches, parallelizes, and caches them.
- Keep heavy derived computation out of the hot path - store what you need and compute the rest at query time.
- Look for accidental work per event: parsing the same static data repeatedly, formatting, or building large intermediate objects.

### Storage writes

**How you know**

```bash
pnpm envio metrics | grep -E 'stalled_on_storage_write|storage_write'
```

`envio_processing_stalled_on_storage_write_seconds` is a large share of the run,
and `envio_storage_write_seconds` is high next to `envio_storage_write_total`
(labeled by `storage`).

**What it means**

The indexer produced entity changes faster than storage accepted them, so
processing paused until the write queue drained.

**What to do**

- Write fewer entities per event. Updating a running-total entity on every event makes each event a write; aggregating in memory and writing on a coarser boundary avoids most of them.
- Don't update the same entity several times within one handler - the last write wins anyway.
- Check your database: [indices](/docs/HyperIndex/database-performance-optimization) speed up reads but every extra index slows writes down, and an under-resourced Postgres shows up here first.
- If [reorg support](/docs/HyperIndex/reorgs-support) is enabled, entity history is written alongside your entities - `envio_rollback_history_prune_seconds` per `entity` shows what that costs.

### External calls

**How you know**

```bash
pnpm envio metrics | grep -E 'envio_effect_'
```

`envio_effect_call_seconds_total` is a large share of the run, or
`envio_effect_queue_wait_seconds` shows calls waiting on a rate limit. Both are
labeled per `effect`, so the slow one names itself.

**What it means**

Handlers are blocked on [Effect API](/docs/HyperIndex/effect-api) calls to
something outside the indexer - an RPC node, an HTTP API, IPFS.

**What to do**

- Set `cache: true` on effects whose result is stable for a given input, so a rerun doesn't repeat the call; `envio_effect_cache` tracks cache size and `envio_effect_cache_invalidations` how often it's discarded.
- Raise the effect's `rateLimit` if the provider allows it - `envio_effect_queue` sitting above zero means calls are queuing rather than running.
- If effects are slow and can't be batched, lower [`full_batch_size`](/docs/HyperIndex/config-schema-reference#fullbatchsize) so a batch doesn't wait on thousands of pending calls.
- Compare `envio_effect_call_seconds_total` with `envio_effect_call_total` to separate "each call is slow" from "there are too many calls".

## Step 4: Prove the fix worked

Benchmarking is only useful as a loop:

1. **Capture a baseline** before changing anything: `pnpm envio metrics > before.txt`.
2. **Change one thing.** Two changes at once make it impossible to attribute the difference.
3. **Re-run over the same block range**, ideally from a fresh database so the run does the same work.
4. **Compare** the same numbers: events per second from step 1, and the share of the run for the counter you targeted.

```bash
pnpm envio metrics > after.txt
diff <(grep -E '^envio_(progress_events|.*_seconds)' before.txt) \
     <(grep -E '^envio_(progress_events|.*_seconds)' after.txt)
```

Two things to watch while you iterate:

- **Memory.** `pnpm envio metrics runtime` reports heap usage, GC, and event-loop lag. An indexer that speeds up while memory climbs is often deferring work rather than avoiding it.
- **Realistic conditions.** Benchmark against a block range with the event mix your indexer actually sees - a quiet range flatters every change you make.

## Next Steps

- [Observability](/docs/HyperIndex/observability) - the full metric reference, logs, health checks, and `_meta` indexing status
- [Preload Optimization](/docs/HyperIndex/preload-optimization) - how batched entity reads work
- [Effect API](/docs/HyperIndex/effect-api) - batching, parallelizing, and caching external calls
- [Database Performance](/docs/HyperIndex/database-performance-optimization) - indices and schema design
- [Latency at the Head](/docs/HyperIndex/latency-at-head) - what to expect once the indexer is caught up
- Still stuck? Share your metrics snapshot in [Discord](https://discord.gg/envio) - we're happy to read it with you.