# Envio docs: HyperSync core > EVM data: quickstart, API tokens, clients, query reference, curl, stream tuning, supported networks, tutorials. Generated at build time from the pages of https://docs.envio.dev. Each page below is the full markdown of one docs page, after a `` line. Append `.md` to that URL to fetch the page on its own. The index of every page is https://docs.envio.dev/llms.txt. Context files, one per product area: - [llms-hyperindex.txt](https://docs.envio.dev/llms-hyperindex.txt): HyperIndex core, about 39k tokens. Build, run and test an indexer: getting started, config.yaml, schema, handlers, CLI, testing, troubleshooting. - [llms-hyperindex-advanced.txt](https://docs.envio.dev/llms-hyperindex-advanced.txt): HyperIndex advanced features, about 31k tokens. Factories, the Effect API, wildcard indexing, preload, contract state, IPFS, HyperSync and RPC sources, reorgs, WebSockets. - [llms-hyperindex-reference.txt](https://docs.envio.dev/llms-hyperindex-reference.txt): HyperIndex performance and reference, about 27k tokens. Config schema reference, observability, performance tuning, benchmarks, supported chains and licensing. - [llms-hyperindex-examples.txt](https://docs.envio.dev/llms-hyperindex-examples.txt): HyperIndex examples and tutorials, about 16k tokens. Example indexers (Uniswap v4, Sablier, Velodrome and Aerodrome) and EVM tutorials. - [llms-hyperindex-migration.txt](https://docs.envio.dev/llms-hyperindex-migration.txt): HyperIndex migration guides, about 13k tokens. Upgrade a V2 indexer to V3, or migrate from The Graph, Ponder or Alchemy, by hand or with AI. - [llms-hyperindex-solana-fuel.txt](https://docs.envio.dev/llms-hyperindex-solana-fuel.txt): HyperIndex on Solana and Fuel, about 20k tokens. Solana programs (instruction and slot handlers, decoding, config) and Fuel contracts. - [llms-envio-cloud.txt](https://docs.envio.dev/llms-envio-cloud.txt): Envio Cloud, about 18k tokens. Deploy, monitor and manage hosted indexers: features, CLI, billing, organisations, self-hosting. - llms-hypersync.txt (this file): HyperSync core, about 32k tokens. EVM data: quickstart, API tokens, clients, query reference, curl, stream tuning, supported networks, tutorials. - [llms-hypersync-solana-fuel.txt](https://docs.envio.dev/llms-hypersync-solana-fuel.txt): HyperSync for Solana and Fuel, about 14k tokens. Solana HyperSync (query schema, client, curl examples) and HyperFuel, HyperSync for the Fuel network. - [llms-hyperrpc.txt](https://docs.envio.dev/llms-hyperrpc.txt): HyperRPC, about 4k tokens. Read-only JSON-RPC endpoint powered by HyperSync: supported methods, usage and supported networks. Pages in this file: - HyperSync: Ultra-Fast Blockchain Data API - HyperSync Quickstart - API Tokens - Using HyperSync - HyperSync Clients - HyperSync Query - Preset Queries - Using curl with HyperSync - Stream Config & Tuning - HyperSync Supported Networks - Troubleshooting - Analyzing All Transactions To and From an Address - Decoding Transaction Traces --- # HyperSync: Ultra-Fast & Flexible Data API > HyperSync is Envio's ultra-fast blockchain data API, a Rust-built alternative to RPC with flexible queries across dozens of EVM chains and Fuel. ## What is HyperSync? HyperSync is a purpose-built data retrieval layer for onchain data, built from the ground up in Rust. It serves as an alternative to traditional JSON-RPC endpoints, letting you filter and select exactly the data you need. On the workloads in our [performance benchmarks](#performance-benchmarks), scans that take hours or days over RPC complete in seconds. :::info HyperSync & HyperIndex **HyperSync** is Envio's high-performance blockchain data engine that serves as a direct replacement for traditional RPC endpoints, delivering up to 2000x faster data access. **HyperIndex** is built on top of HyperSync, providing a complete indexing framework with schema management, event handling, and GraphQL APIs. Use HyperSync directly when you need raw blockchain data at maximum speed, or use HyperIndex when you need a full-featured indexing solution. ::: ## The Problem HyperSync Solves Traditional blockchain data access through JSON-RPC faces several limitations: - **Speed constraints**: Retrieving large amounts of historical data can take days - **Query flexibility**: Complex data analysis requires many separate calls - **Cost inefficiency**: Expensive for data-intensive applications ## Key Benefits - **Exceptional Performance**: Retrieve and process blockchain data up to 1000x faster than traditional RPC methods - **Comprehensive Coverage**: Access data across [81+ EVM chains](https://docs.envio.dev/docs/HyperSync/hypersync-supported-networks) and Fuel, with new networks added regularly - **Flexible Query Capabilities**: Filter, select, and process exactly the data you need with powerful query options - **Cost Efficiency**: Dramatically reduce infrastructure costs for data-intensive applications - **Simple Integration**: Client libraries available for Python, Rust, Node.js, and Go ## Performance Benchmarks HyperSync delivers transformative performance compared to traditional methods: | Task | Traditional RPC | HyperSync | Improvement | | --- | --- | --- | --- | | Scan Arbitrum blockchain for sparse log data | Hours/Days | 2 seconds | ~2000x faster | | Fetch all Uniswap v3 PoolCreated events ethereum | Hours | Seconds | ~500x faster | ## Use Cases HyperSync powers a wide range of blockchain applications, enabling developers to build tools that would be impractical with traditional data access methods: ### General Applications - **Blockchain Indexers**: Build high-performance data indexers with minimal infrastructure - **Data Analytics**: Perform complex on-chain analysis in seconds instead of days - **Block Explorers**: Create responsive explorers with comprehensive data access - **Monitoring Tools**: Track blockchain activity with near real-time updates - **Cross-chain Applications**: Access unified data across multiple networks - **ETL Pipelines**: Create pipelines to extract and save data fast ### Powered by HyperSync #### [HyperIndex](https://docs.envio.dev/docs/HyperIndex/overview) - **100x faster blockchain indexing** across 81+ EVM chains and Fuel - **Powers 100 plus applications** like v4.xyz analytics #### [ChainDensity.xyz](https://chaindensity.xyz) - **Fast transaction/event density analysis** for any address - **Generates insights in seconds** that would take hours with traditional methods #### [Scope.sh](https://scope.sh) - **Ultra-fast Account Abstraction (AA) focused block explorer** - **Fast historical data retrieval** with minimal latency #### [LogTUI](https://www.npmjs.com/package/logtui) - **Terminal-based UI** for finding all historical blockchain events - **Built-in presets** for 20+ protocols (Uniswap, Chainlink, Aave, ENS, etc.) - **Try it**: `pnpx logtui aave arbitrum` to track Aave events on Arbitrum in your terminal ## See HyperSync in Action [Video](https://www.youtube.com/watch?v=iu_469ELotw) ## Next Steps - **[Try the Quick Start Guide](https://docs.envio.dev/docs/HyperSync/hypersync-quickstart)** to get up and running in minutes - **[Build queries visually](http://builder.hypersync.xyz)** with our Intuitive Query Builder - [Get an API Token](https://docs.envio.dev/docs/HyperSync/api-tokens) to access HyperSync services - [View Supported Networks](https://docs.envio.dev/docs/HyperSync/hypersync-supported-networks) to see available chains - [Check Client Documentation](https://docs.envio.dev/docs/HyperSync/hypersync-clients) for language-specific guides - [Join our Discord](https://discord.gg/envio) for support and updates :::note Our documentation is continuously improving! If you have questions or need assistance, please reach out in our [Discord community](https://discord.gg/envio). ::: --- # HyperSync Quickstart > Get started quickly with HyperSync to stream and filter blockchain events. :::tip Easiest way to get started: the Query Builder The **[HyperSync Query Builder](https://builder.hypersync.xyz)** lets you construct and run queries directly in your browser β€” no install, no code required. It's the fastest way to get familiar with HyperSync and see what's possible. **[Open builder.hypersync.xyz β†’](https://builder.hypersync.xyz)** ::: Get up and running with HyperSync in minutes. This guide will help you start accessing blockchain data at unprecedented speeds with minimal setup. For a conceptual overview before you dive in, see [What is HyperSync?](https://docs.envio.dev/blog/what-is-hypersync). ## Quickest Start: Try LogTUI Want to see HyperSync in action with zero setup? Try LogTUI, a terminal-based blockchain event viewer: ```bash # Monitor Aave events on Arbitrum (no installation needed) pnpx logtui aave arbitrum ``` ## Clone the Quickstart Repository The fastest way to get started is to clone our minimal example repository: ```bash git clone https://github.com/enviodev/hypersync-quickstart.git cd hypersync-quickstart ``` This repository contains everything you need to start streaming blockchain data using HyperSync. ## Install Dependencies ```bash # Using pnpm (recommended) pnpm install ``` ## Choose Your Adventure The repository includes three different script options, all of which retrieve Uniswap V3 events from Ethereum mainnet: ```bash # Run minimal version (recommended for beginners) node run-simple.js # Run full version with progress bar node run.js # Run version with terminal UI node run-tui.js ``` That's it! You're now streaming data directly from Ethereum mainnet through HyperSync! (TUI version below) ![HyperSync in Action](https://docs.envio.dev/img/hypersync.gif) ## Understanding the Code Let's look at the core concepts in the example code: ### 1. Initialize the Client ```typescript import { HypersyncClient } from "@envio-dev/hypersync-client"; // Initialize Hypersync client const client = new HypersyncClient({ url: "https://eth.hypersync.xyz", // Change this URL for different networks apiToken: process.env.ENVIO_API_TOKEN!, }); ``` > **Note:** To connect to different networks, see the [Supported Networks](https://docs.envio.dev/docs/HyperSync/hypersync-supported-networks) page for a complete list of available URLs. ### 2. Build Your Query The heart of HyperSync is the query object, which defines what data you want to retrieve: ```javascript let query = { fromBlock: 0, // Start block (0 = genesis) logs: [ // Filter for specific events { topics: [topic0_list], // Event signatures we're interested in }, ], fieldSelection: { // Only return fields we need log: [ "Data", "Address", "Topic0", "Topic1", "Topic2", "Topic3", ], }, }; ``` ### 3. Stream and Process Results ```javascript // Start streaming events const stream = await client.stream(query, {}); let totalEvents = 0; while (true) { const res = await stream.recv(); // Exit when the stream reaches the end if (res === null) break; // Process results if (res.data && res.data.logs) { // Do something with the logs totalEvents += res.data.logs.length; } // Update starting block for next batch if (res.nextBlock) { query.fromBlock = res.nextBlock; } } ``` ## Key Concepts for Building Queries ### Filtering Data HyperSync lets you filter blockchain data in several ways: - **Log filters**: Find specific events by contract address and event signature - **Transaction filters**: Filter by sender/receiver addresses, method signatures, etc. - **Trace filters**: Access internal transactions and state changes (only supported on select networks like Ethereum Mainnet) - **Block filters**: Get data from specific block ranges ### Field Selection One of HyperSync's most powerful features is the ability to retrieve only the fields you need: ```javascript fieldSelection: { // Block fields block: ["Number", "Timestamp"], // Log fields log: ["Address", "Topic0", "Data"], // Transaction fields transaction: ["From", "To", "Value"], } ``` This selective approach dramatically reduces unnecessary data transfer and improves performance. ### Join Modes HyperSync allows you to control how related data is joined: - **JoinNothing**: Return only exact matches - **JoinAll**: Return matches plus all related objects - **Default**: Return a reasonable set of related objects ## Examples ### Finding Uniswap V3 Events This example (from the quickstart repo) streams all Uniswap V3 events from the beginning of Ethereum: ```typescript import { keccak256, toHex } from "viem"; import { HypersyncClient } from "@envio-dev/hypersync-client"; // Define Uniswap V3 event signatures const event_signatures = [ "PoolCreated(address,address,uint24,int24,address)", "Burn(address,int24,int24,uint128,uint256,uint256)", "Initialize(uint160,int24)", "Mint(address,address,int24,int24,uint128,uint256,uint256)", "Swap(address,address,int256,int256,uint160,uint128,int24)", ]; // Create topic0 hashes from event signatures const topic0_list = event_signatures.map((sig) => keccak256(toHex(sig))); // Initialize Hypersync client const client = new HypersyncClient({ url: "https://eth.hypersync.xyz", apiToken: process.env.ENVIO_API_TOKEN!, }); // Define query for Uniswap V3 events let query = { fromBlock: 0, logs: [ { topics: [topic0_list], }, ], fieldSelection: { log: [ "Data", "Address", "Topic0", "Topic1", "Topic2", "Topic3", ], }, }; const main = async () => { console.log("Starting Uniswap V3 event scan..."); const stream = await client.stream(query, {}); // Process stream... }; main(); ``` ## Supported Networks HyperSync supports 81+ EVM-compatible networks. You can change networks by simply changing the client URL: ```typescript // Ethereum Mainnet const ethClient = new HypersyncClient({ url: "https://eth.hypersync.xyz", apiToken: process.env.ENVIO_API_TOKEN!, }); // Arbitrum const arbitrumClient = new HypersyncClient({ url: "https://arbitrum.hypersync.xyz", apiToken: process.env.ENVIO_API_TOKEN!, }); // Base const baseClient = new HypersyncClient({ url: "https://base.hypersync.xyz", apiToken: process.env.ENVIO_API_TOKEN!, }); ``` See the [Supported Networks](https://docs.envio.dev/docs/HyperSync/hypersync-supported-networks) page for a complete list. ## Using LogTUI This quickstart repository powers [LogTUI](https://www.npmjs.com/package/logtui), a terminal-based blockchain event viewer built on HyperSync. LogTUI lets you monitor events from popular protocols across multiple chains with zero configuration. Try it with a single command: ```bash # Monitor Uniswap events on unichain pnpx logtui uniswap-v4 unichain # Monitor Aave events on Arbitrum pnpx logtui aave arbitrum # See all available options pnpx logtui --help ``` LogTUI supports scanning historically for any events across all networks supported by HyperSync. ## Next Steps You're now ready to build with HyperSync! Here are some resources for diving deeper: - [Client Libraries](https://docs.envio.dev/docs/HyperSync/hypersync-clients) - Explore language-specific clients - [Query Reference](https://docs.envio.dev/docs/HyperSync/hypersync-query) - Learn advanced query techniques - **[Build queries visually](http://builder.hypersync.xyz)** - Use our Intuitive Query Builder - [curl Examples](https://docs.envio.dev/docs/HyperSync/hypersync-curl-examples) - Test queries directly in your terminal - [Complete Getting Started Guide](https://docs.envio.dev/docs/HyperSync/hypersync-usage) - More comprehensive guidance ## API Token An API token is required to use HyperSync. [Get an API token](https://docs.envio.dev/docs/HyperSync/api-tokens) and set it as an environment variable: ```bash export ENVIO_API_TOKEN="your-api-token-here" ``` Congratulations! You've taken your first steps with HyperSync, bringing ultra-fast blockchain data access to your applications. Happy building! --- # API tokens for HyperSync > Learn how to generate and use HyperSync API tokens for secure access. HyperSync and HyperRPC need an API token. Requests without one are rejected with HTTP 401. :::info Envio Cloud Indexers deployed to Envio Cloud have their own access to HyperSync and don't need a token. ::: ## Generating API tokens Create tokens in the [Envio Dashboard](https://envio.dev/app/api-tokens). The steps below are for HyperSync. HyperRPC tokens live on their own **HyperRPC Tokens** page in the same sidebar. 1. Go to the dashboard and sign in, or create an account. You land on the **HyperSync tokens** page for your personal organisation. 2. If the token is for a team, pick that organisation under **Organisations** in the left sidebar. 3. Under **Tokens by package**, click a package row, for example **Free package**, to expand it. The token form only shows once a package is expanded. 4. At the bottom of the expanded package, type a name in the **Token name** field, for example `Development` or `Production`. 5. Click **Create**. A **Token created** window opens with your new token. 6. Click the copy button next to the token and save it somewhere safe, such as your project's `.env` file. ![Creating a HyperSync API token in the Envio Dashboard](https://docs.envio.dev/docs-assets/create-hypersync-api-token.gif) :::warning Copy your token before closing the window Envio does not keep the secret on its servers. For a token in a paid package, the **Token created** window is the only place you can copy it. After you close it, the list only shows the last four characters, and if you lose the token you need to delete it with the bin icon and create a new one. A token in a free package can be revealed and copied again from the list. ::: ### Token limits and product access Personal accounts get one free token. If the **Token name** field says **Token limit reached**, the package has used all its tokens. Use the one you have, delete it and create a new one, or click **Manage subscriptions** to add a paid package. Tokens in new paid packages work for one product, HyperSync or HyperRPC. Tokens created before the billing system launched work for both. ## Adding the token to your project For HyperIndex, put the token in the `.env` file at the root of your project. HyperIndex reads it from the `ENVIO_API_TOKEN` variable. ```bash ENVIO_API_TOKEN=your_token_here ``` For HyperSync client scripts, the examples in [Using the token in code](#using-the-token-in-code) read the same `ENVIO_API_TOKEN` variable. Export it in your shell, or load your `.env` file with your usual tool. ### Letting `envio init` add it for you If you give `envio init` a token, it writes it into the new project's `.env` file. Pass it with the `envio init` option `--api-token`, or set `ENVIO_API_TOKEN` in your shell first. This suits scripted or agent-driven setups. ```bash ENVIO_API_TOKEN=your_token_here pnpx envio init template \ --template erc20 --language typescript \ --name my-indexer --directory my-indexer ``` The `.env` file it creates looks like this. ```bash # To create or update a token visit https://envio.dev/app/api-tokens ENVIO_API_TOKEN="your_token_here" ``` ![Running envio init with a token and checking the .env file it writes](https://docs.envio.dev/docs-assets/envio-init-api-token.gif) ## Using the token in code Pass the token when you create a HyperSync client. **TypeScript/JavaScript** ```typescript const client = new HypersyncClient({ url: "https://eth.hypersync.xyz", apiToken: process.env.ENVIO_API_TOKEN!, }); ``` **Python** ```python import os import hypersync client = hypersync.HypersyncClient(hypersync.ClientConfig( url="https://eth.hypersync.xyz", api_token=os.environ.get("ENVIO_API_TOKEN") )) ``` **Rust** ```rust let client = Client::new(ClientConfig { url: "https://eth.hypersync.xyz".to_string(), api_token: std::env::var("ENVIO_API_TOKEN").expect("ENVIO_API_TOKEN must be set"), ..Default::default() }).unwrap(); ``` ## Understanding usage On the [HyperSync tokens](https://envio.dev/app/api-tokens) page, expand a package to see its rate limit usage. Click a token's row to see usage statistics for that token. ![Checking HyperSync token usage in the Envio Dashboard](https://docs.envio.dev/docs-assets/check-hypersync-token-usage.gif) ## Rate limits The limits below apply to HyperSync. ### How is the rate limit applied? Each token has a budget per 60-second window. The budget is shared across chains, so requests to `eth.hypersync.xyz` and `base.hypersync.xyz` with the same token draw from the same budget. The rate limit caps how fast you can query, not how much you can query in a month. ### What limit does my plan have? Each plan has its own rate limit. See [HyperSync pricing](https://envio.dev/pricing/hypersync) for the current limits and overage options. ### What happens when I hit the limit? The server responds with HTTP 429 until the window resets. When streaming, the HyperSync clients wait for the reset and retry, so a stream slows down rather than failing. Successful and 429 responses carry `x-ratelimit-*` headers with your remaining budget and seconds until reset. The budget is counted in units rather than requests. See [Inspecting rate limits from your code](https://docs.envio.dev/docs/HyperSync/stream-config-tuning.md#inspecting-rate-limits-from-your-code) to read them. ### How do I make fewer requests? Lower `concurrency` in your stream config so fewer requests run in parallel. See [Stream Config & Tuning](https://docs.envio.dev/docs/HyperSync/stream-config-tuning.md#rate-limits). For more headroom, move to a plan with a higher limit. ## Common errors | Error | Cause | Fix | | --- | --- | --- | | HTTP 401, `Your token is malformed` | No token was sent, or the token is not in the expected format | Create a token in the [Envio Dashboard](https://envio.dev/app/api-tokens) and pass it as shown in [Using the token in code](#using-the-token-in-code) | | HTTP 403, `Your token is unknown or pending activation` | The token does not exist or is not active | Check the token was copied correctly and is still listed and active in the [Envio Dashboard](https://envio.dev/app/api-tokens) | | HTTP 403, `Your token does not have access to this product` | The token does not have access to the product you called, for example HyperRPC | Check which package the token belongs to in the [Envio Dashboard](https://envio.dev/app/api-tokens) or see [HyperRPC pricing](https://envio.dev/pricing/hyperrpc) | | HTTP 429 | The token used its budget for the current window | Wait for the window to reset, lower `concurrency`, or move to a higher plan | ## Security best practices - **Never commit tokens to git.** Keep them in environment variables or a `.env` file, not in your code. - **Add `.env` to your `.gitignore`** so the token never gets committed by accident. - **Rotate tokens now and then.** Create a new token, switch your apps over, then delete the old one. - **Share tokens only with people who need them.** --- # Getting Started with HyperSync > Learn how to fetch, filter, and decode blockchain data using HyperSync. :::tip Easiest way to get started: the Query Builder The **[HyperSync Query Builder](https://builder.hypersync.xyz)** lets you construct and run queries directly in your browser β€” no install, no code required. It's the fastest way to get familiar with HyperSync and see what's possible. **[Open builder.hypersync.xyz β†’](https://builder.hypersync.xyz)** ::: HyperSync is Envio's high-performance blockchain data engine that provides up to 2000x faster access to blockchain data compared to traditional RPC endpoints. This guide will help you understand how to effectively use HyperSync in your applications. ## Quick Start Video Watch this quick tutorial to see HyperSync in action: [Video](https://www.youtube.com/watch?v=S9Z6XkY3aP8) ## Core Concepts HyperSync revolves around two main concepts: 1. **Queries** - Define what blockchain data you want to retrieve 2. **Output Configuration** - Specify how you want that data formatted and delivered Think of queries as your data filter and the output configuration as your data processor. ## Building Effective Queries Queries are the heart of working with HyperSync. They allow you to filter for specific blocks, logs, transactions, and traces. ### Query Structure A basic HyperSync query contains: ```python query = hypersync.Query( from_block=12345678, # Required: Starting block number to_block=12345778, # Optional: Ending block number field_selection=field_selection, # Required: What fields to return logs=[log_selection], # Optional: Filter for specific logs transactions=[tx_selection], # Optional: Filter for specific transactions traces=[trace_selection], # Optional: Filter for specific traces include_all_blocks=False, # Optional: Include blocks with no matches max_num_blocks=1000, # Optional: Limit number of blocks processed max_num_transactions=5000, # Optional: Limit number of transactions processed max_num_logs=5000, # Optional: Limit number of logs processed max_num_traces=5000 # Optional: Limit number of traces processed ) ``` ### Field Selection Field selection allows you to specify exactly which data fields you want to retrieve. This improves performance by only fetching what you need: ```python field_selection = hypersync.FieldSelection( # Block fields you want to retrieve block=[ BlockField.NUMBER, BlockField.TIMESTAMP, BlockField.HASH ], # Transaction fields you want to retrieve transaction=[ TransactionField.HASH, TransactionField.FROM, TransactionField.TO, TransactionField.VALUE ], # Log fields you want to retrieve log=[ LogField.ADDRESS, LogField.TOPIC0, LogField.TOPIC1, LogField.TOPIC2, LogField.TOPIC3, LogField.DATA, LogField.TRANSACTION_HASH ], # Trace fields you want to retrieve (if applicable) trace=[ TraceField.FROM, TraceField.TO, TraceField.VALUE ] ) ``` ### Filtering for Specific Data For most use cases, you'll want to filter for specific logs, transactions, or traces: #### Log Selection Example ```python # Filter for Transfer events from USDC contract log_selection = hypersync.LogSelection( address=["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"], # USDC contract topics=[ ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"] # Transfer event signature ] ) ``` #### Transaction Selection Example ```python # Filter for transactions to the Uniswap V3 router tx_selection = hypersync.TransactionSelection( to=["0xE592427A0AEce92De3Edee1F18E0157C05861564"] # Uniswap V3 Router ) ``` ## Processing the Results HyperSync provides multiple ways to process query results: ### Stream to Parquet Files Parquet is the recommended format for large data sets: ```python # Configure output format parquet_config = hypersync.StreamConfig( hex_output=hypersync.HexOutput.PREFIXED, event_signature="Transfer(address indexed from, address indexed to, uint256 value)" ) # Stream results to a Parquet file await client.collect_parquet("data_directory", query, parquet_config) ``` `event_signature` is accepted by the parquet and arrow methods, `collect_parquet` (which writes an extra `decoded_logs.parquet` alongside the raw output), `collect_arrow` and `stream_arrow`. The methods returning plain Python objects reject it: `collect`, `collect_events`, `stream` and `stream_events` all raise `config.event_signature can't be passed to simple type function`. With those, decode explicitly instead, as shown below. ### Process Data in Memory For immediate processing, stream the results and decode the logs yourself: ```python # Build a decoder for the events you care about decoder = hypersync.Decoder([ "Transfer(address indexed from, address indexed to, uint256 value)" ]) stream = await client.stream(query, hypersync.StreamConfig()) while True: res = await stream.recv() if res is None: # reached the end of the requested range break for log in await decoder.decode_logs(res.data.logs): if log is None: # a log that doesn't match the signature continue # Indexed parameters land in `indexed`, the rest in `body` print(f"Transfer from {log.indexed[0].val} to {log.indexed[1].val} of {log.body[0].val}") ``` ## Tips and Best Practices ### Performance Optimization - **Use Appropriate Batch Sizes**: Adjust batch size based on your chain and use case: ```python config = hypersync.StreamConfig( hex_output=hypersync.HexOutput.PREFIXED, batch_size=1000000, # Initial block range for the first wave of requests concurrency=10, # Use 10 concurrent workers ) ``` - **Enable Trace Logs**: Set `RUST_LOG=trace` to see detailed progress: ```bash export RUST_LOG=trace ``` - **Paginate Large Queries**: A single `get` request has a 5-second time limit, so it may return only part of the range. The `collect` and `stream` methods handle this for you. If you call `get` directly, use `next_block` to continue from where it stopped. ```python query.from_block = start_block query.to_block = end_block while True: res = await client.get(query) # process res.data here if res.next_block >= min(end_block, res.archive_height): break query.from_block = res.next_block ``` ### Network-Specific Considerations - **High-Volume Networks**: For networks like Ethereum Mainnet, use smaller block ranges or more specific filters - **Low-Volume Networks**: For smaller chains, you can process the entire chain in one query ## Complete Example Here's a complete example that fetches all USDC Transfer events: ```python import hypersync from hypersync import ( LogSelection, LogField, BlockField, FieldSelection, TransactionField, HexOutput ) import asyncio async def collect_usdc_transfers(): # Initialize client client = hypersync.HypersyncClient( hypersync.ClientConfig( url="https://eth.hypersync.xyz", api_token="your-token-here", # Get from https://docs.envio.dev/docs/HyperSync/api-tokens ) ) # Define field selection field_selection = hypersync.FieldSelection( block=[BlockField.NUMBER, BlockField.TIMESTAMP], transaction=[TransactionField.HASH], log=[ LogField.ADDRESS, LogField.TOPIC0, LogField.TOPIC1, LogField.TOPIC2, LogField.DATA, ] ) # Define query for USDC transfers query = hypersync.Query( from_block=12000000, to_block=12100000, field_selection=field_selection, logs=[ LogSelection( address=["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"], # USDC contract topics=[ ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"] # Transfer signature ] ) ] ) # Configure output config = hypersync.StreamConfig( hex_output=HexOutput.PREFIXED, event_signature="Transfer(address indexed from, address indexed to, uint256 value)" ) # Collect data to a Parquet file await client.collect_parquet("usdc_transfers", query, config) print(f"Processed blocks {query.from_block} to {query.to_block}") asyncio.run(collect_usdc_transfers()) ``` ## Decoding Event Logs When working with blockchain data, event logs contain encoded data that needs to be properly decoded to extract meaningful information. HyperSync provides powerful decoding capabilities to simplify this process. ### Understanding Log Structure Event logs in Ethereum have the following structure: - **Address**: The contract that emitted the event - **Topic0**: The event signature hash (keccak256 of the event signature) - **Topics 1-3**: Indexed parameters (up to 3) - **Data**: Non-indexed parameters packed together ### Using the Decoder HyperSync's client libraries include a `Decoder` class that can parse these raw logs into structured data: ```javascript // Create a decoder with event signatures const decoder = Decoder.fromSignatures([ "Transfer(address indexed from, address indexed to, uint256 amount)", "Approval(address indexed owner, address indexed spender, uint256 amount)", ]); // Decode logs const decodedLogs = await decoder.decodeLogs(logs); ``` ### Single vs. Multiple Event Types HyperSync provides flexibility to decode different types of event logs: - **Single Event Type**: For processing one type of event (e.g., only Swap events) - See complete example: [run-decoder.js](https://github.com/enviodev/hypersync-quickstart/blob/main/run-decoder.js) - **Multiple Event Types**: For processing different events from the same contract (e.g., Transfer and Approval) - See complete example: [run-decoder-multi.js](https://github.com/enviodev/hypersync-quickstart/blob/main/run-decoder-multi.js) ### Working with Decoded Data After decoding, you can access the log parameters in a structured way: - **Indexed parameters**: Available in `decodedLog.indexed` array - **Non-indexed parameters**: Available in `decodedLog.body` array Each parameter object has a single field, `val`, which holds the decoded value. For example, to access parameters from a Transfer event: ```javascript // Access indexed parameters (from, to) const from = decodedLog.indexed[0]?.val.toString(); const to = decodedLog.indexed[1]?.val.toString(); // Access non-indexed parameters (amount) const amount = decodedLog.body[0]?.val.toString(); ``` ### Benefits of Using the Decoder - **Type Safety**: Values are properly converted to their corresponding types - **Simplified Access**: Direct access to decoded values by position - **Batch Processing**: Decode multiple logs with a single call - **Multiple Event Support**: Handle different event types in the same processing pipeline ## Next Steps Now that you understand the basics of using HyperSync: - Browse the [Python Client](https://docs.envio.dev/docs/HyperSync/hypersync-clients.md) or other language-specific clients - Learn about [advanced query options](https://docs.envio.dev/docs/HyperSync/hypersync-query.md) - Tune throughput, and read your rate-limit quota from code, in [Stream Config & Tuning](https://docs.envio.dev/docs/HyperSync/stream-config-tuning.md) - See [example queries for common use cases](https://docs.envio.dev/docs/HyperSync/hypersync-curl-examples.md) - [Get your API token](https://docs.envio.dev/docs/HyperSync/api-tokens.md) to start building For detailed API references and examples in other languages, check our [client documentation](https://docs.envio.dev/docs/HyperSync/hypersync-clients.md). --- # HyperSync Client Libraries > Explore HyperSync clients for fast blockchain data access in Node.js, Python, Rust, and Go. HyperSync provides powerful client libraries that enable you to integrate high-performance blockchain data access into your applications. These libraries handle the communication with HyperSync servers, data serialization/deserialization, and provide convenient APIs for querying blockchain data. ## Quick Links | Client | Resources | | --- | --- | | **Node.js** | [πŸ“ API Docs](https://enviodev.github.io/hypersync-client-node/) Β· [πŸ“¦ NPM](https://www.npmjs.com/package/@envio-dev/hypersync-client) Β· [πŸ’» GitHub](https://github.com/enviodev/hypersync-client-node) Β· [πŸ§ͺ Examples](https://github.com/enviodev/hypersync-client-node/tree/main/examples) | | **Python** | [πŸ“¦ PyPI](https://pypi.org/project/hypersync/) Β· [πŸ’» GitHub](https://github.com/enviodev/hypersync-client-python) Β· [πŸ§ͺ Examples](https://github.com/enviodev/hypersync-client-python/tree/main/examples) | | **Rust** | [πŸ“¦ Crates.io](https://crates.io/crates/hypersync-client) Β· [πŸ“ API Docs](https://docs.rs/hypersync-client/latest/hypersync_client/) Β· [πŸ’» GitHub](https://github.com/enviodev/hypersync-client-rust) Β· [πŸ§ͺ Examples](https://github.com/enviodev/hypersync-client-rust/tree/main/examples) | | **Go** _(community)_ | [πŸ’» GitHub](https://github.com/enviodev/hypersync-client-go) Β· [πŸ§ͺ Examples](https://github.com/enviodev/hypersync-client-go/tree/main/examples) | | **Solana** _(Rust)_ | [πŸ“– Guide](https://docs.envio.dev/docs/HyperSync/solana-client.md) Β· [πŸ“¦ Crates.io](https://crates.io/crates/hypersync-client-solana) Β· [πŸ“ API Docs](https://docs.rs/hypersync-client-solana) Β· [πŸ’» GitHub](https://github.com/enviodev/hypersync-client-solana) | | **API Tokens** | [πŸ”‘ Get Tokens](https://docs.envio.dev/docs/HyperSync/api-tokens.md) | ## Client Overview All HyperSync clients share these key features: - **High Performance**: Built on a common Rust foundation for maximum efficiency - **Optimized Transport**: Uses binary formats to minimize bandwidth and maximize throughput - **Consistent Experience**: Similar APIs across all language implementations - **Automatic Pagination**: Handles large data sets efficiently - **Event Decoding**: Parses binary event data into structured formats Choose the client that best matches your application's technology stack: | Feature | Node.js | Python | Rust | Go | | --- | --- | --- | --- | --- | | Async Support | βœ… | βœ… | βœ… | βœ… | | Typing | TypeScript | Type Hints | Native | Native | | Data Formats | JSON, Parquet | JSON, Parquet, CSV | JSON, Parquet | JSON, Parquet | | Memory Efficiency | Good | Better | Best | Better | | Installation | npm | pip | cargo | go get | ## Node.js Client :::info Resources - [πŸ“ API Documentation](https://enviodev.github.io/hypersync-client-node/) - [πŸ“¦ NPM Package](https://www.npmjs.com/package/@envio-dev/hypersync-client) - [πŸ’» GitHub Repository](https://github.com/enviodev/hypersync-client-node) - [πŸ§ͺ Example Projects](https://github.com/enviodev/hypersync-client-node/tree/main/examples) ::: The Node.js client provides a TypeScript-first experience for JavaScript developers. ### Installation ```bash # Using npm npm install @envio-dev/hypersync-client # Using yarn yarn add @envio-dev/hypersync-client # Using pnpm pnpm add @envio-dev/hypersync-client ``` ## Python Client :::info Resources - [πŸ“¦ PyPI Package](https://pypi.org/project/hypersync/) - [πŸ’» GitHub Repository](https://github.com/enviodev/hypersync-client-python) - [πŸ§ͺ Example Projects](https://github.com/enviodev/hypersync-client-python/tree/main/examples) ::: The Python client provides a Pythonic interface with full type hinting support. ### Installation ```bash pip install hypersync ``` ## Rust Client :::info Resources - [πŸ“¦ Crates.io Package](https://crates.io/crates/hypersync-client) - [πŸ“ API Documentation](https://docs.rs/hypersync-client/latest/hypersync_client/) - [πŸ’» GitHub Repository](https://github.com/enviodev/hypersync-client-rust) - [πŸ§ͺ Example Projects](https://github.com/enviodev/hypersync-client-rust/tree/main/examples) ::: The Rust client provides the most efficient and direct access to HyperSync, with all the safety and performance benefits of Rust. ### Installation Add the following to your `Cargo.toml`: ```toml [dependencies] hypersync-client = "1" tokio = { version = "1", features = ["full"] } ``` ## Solana Client :::info Resources - [πŸ“– Solana Client Guide](https://docs.envio.dev/docs/HyperSync/solana-client.md) - [πŸ“¦ Crates.io Package](https://crates.io/crates/hypersync-client-solana) - [πŸ“ API Documentation](https://docs.rs/hypersync-client-solana) - [πŸ’» GitHub Repository](https://github.com/enviodev/hypersync-client-solana) ::: Solana HyperSync has its own client, because the query and table model is Solana-shaped (slots, instruction calls, account activity) rather than EVM-shaped. It is Rust today, with unpublished Node bindings in the same repository. Everything client-side is covered in the [Solana client guide](https://docs.envio.dev/docs/HyperSync/solana-client.md). ```toml [dependencies] hypersync-client-solana = "0.2" ``` ## Go Client :::info Resources - [πŸ’» GitHub Repository](https://github.com/enviodev/hypersync-client-go) - [πŸ§ͺ Example Projects](https://github.com/enviodev/hypersync-client-go/tree/main/examples) ::: :::caution Community Maintained The Go client is community maintained and marked as work-in-progress. For production use, you may want to test thoroughly or consider the officially supported clients. ::: The Go client provides a native Go interface for accessing HyperSync, with support for streaming and decoding blockchain data. ### Installation ```bash go get github.com/enviodev/hypersync-client-go ``` ## Using API Tokens :::info Get Your API Token You'll need an API token to use any HyperSync client. [Get your token here](https://docs.envio.dev/docs/HyperSync/api-tokens.md). ::: All HyperSync clients require an API token for authentication. Tokens are used to manage access and usage limits. To get an API token: 1. Visit [Envio](https://envio.dev) 2. Register or sign in with your github account 3. Navigate to the API Tokens section 4. Create a new token For detailed instructions, see our [API Tokens](https://docs.envio.dev/docs/HyperSync/api-tokens.md) guide. ## Client Selection Guide Choose the client that best fits your use case: :::tip Node.js Client **Choose when**: You're building JavaScript/TypeScript applications or if your team is most comfortable with the JavaScript ecosystem. ::: :::tip Python Client **Choose when**: You're doing data science work, need integration with pandas/numpy, or if your team prefers Python's simplicity. ::: :::tip Rust Client **Choose when**: You need maximum performance, are doing systems programming, or building performance-critical applications. ::: :::tip Go Client **Choose when**: You're working in a Go ecosystem and want native integration with Go applications. Note that this client is community maintained. ::: ## Additional Resources - [πŸ“š HyperSync Usage Guide](https://docs.envio.dev/docs/HyperSync/hypersync-usage.md) - [πŸ“ Query Reference](https://docs.envio.dev/docs/HyperSync/hypersync-query.md) - [πŸ§ͺ cURL Examples](https://docs.envio.dev/docs/HyperSync/hypersync-curl-examples.md) - [πŸ“Š Supported Networks](https://docs.envio.dev/docs/HyperSync/hypersync-supported-networks.md) - [🧱 HyperIndex blockchain indexer](https://docs.envio.dev/docs/HyperIndex/overview), the full framework with schema, handlers, and a hosted GraphQL API, powered by HyperSync ## Support Need help getting started or have questions about our clients? Connect with our community: - [Discord Community](https://discord.gg/envio) - [GitHub Issues](https://github.com/enviodev) ``` ``` --- # HyperSync Query > Explore HyperSync queries to efficiently retrieve, filter, and join blockchain data. This guide explains how to structure queries for HyperSync to efficiently retrieve blockchain data. You'll learn both the basics and advanced techniques to make the most of HyperSync's powerful querying capabilities. :::note HyperFuel Limitations Not all features implemented in HyperSync are available in HyperFuel (the Fuel implementation of HyperSync). For example, as of this writing, stream and collect functions aren't implemented in the Fuel client. ::: ## Client Examples HyperSync offers client libraries in multiple languages, each with its own comprehensive examples. Instead of providing generic examples here, we recommend exploring the language-specific examples: | Client | Example Links | | --- | --- | | **Node.js** | [Example Repository](https://github.com/enviodev/hypersync-client-node/tree/main/examples) | | **Python** | [Example Repository](https://github.com/enviodev/hypersync-client-python/tree/main/examples) | | **Rust** | [Example Repository](https://github.com/enviodev/hypersync-client-rust/tree/main/examples) | | **Go** | [Example Repository](https://github.com/enviodev/hypersync-client-go/tree/main/examples) | Additionally, we maintain a comprehensive collection of real-world examples covering various use cases across different languages: - [**HyperSync Examples**](https://github.com/enviodev/hypersync-examples) - A diverse collection of practical examples demonstrating HyperSync's capabilities in Python, JavaScript, TypeScript, Rust, and more. For more details on client libraries, see the [HyperSync Clients](https://docs.envio.dev/docs/HyperSync/hypersync-clients.md) documentation. :::tip Visual Query Builder Need help building queries? Try our **[Intuitive Query Builder](http://builder.hypersync.xyz)** to construct queries visually and see the results in real-time. ::: :::tip Developer Tip Set the `RUST_LOG` environment variable to `trace` for more detailed logs when using client libraries. ::: ## Table of Contents 1. [Understanding HyperSync Queries](#understanding-hypersync-queries) 2. [Query Execution Process](#query-execution-process) 3. [Query Structure Reference](#query-structure-reference) 4. [Data Schema](#data-schema) 5. [Response Structure](#response-structure) 6. [Stream and Collect Functions](#stream-and-collect-functions) 7. [Working with Join Modes](#working-with-join-modes) 8. [Best Practices](#best-practices) ## Understanding HyperSync Queries A HyperSync query defines what blockchain data you want to retrieve and how you want it returned. Unlike regular RPC calls, HyperSync queries offer: - **Flexible filtering** across logs, transactions, traces, and blocks - **Field selection** to retrieve only the data you need - **Automatic pagination** to handle large result sets - **Join capabilities** that link related blockchain data together ### Core Concepts - **Selections**: Define criteria for filtering blockchain data (logs, transactions, traces) - **Field Selection**: Specify which fields to include in the response - **Limits**: Control query execution time and response size - **Joins**: Determine how related data is connected in the response ## Query Execution Process ### How Data is Organized HyperSync organizes blockchain data into groups of contiguous blocks. When executing a query: 1. The server identifies which block group contains the starting block 2. It processes data groups sequentially until it hits a limit 3. Results are returned along with a `next_block` value for pagination ### Query Limits HyperSync enforces several types of limits to ensure efficient query execution: | Limit Type | Description | Behavior | | --- | --- | --- | | **Time** | Server-configured maximum execution time | May slightly exceed limit to complete current block group | | **Response Size** | Maximum data returned | May slightly exceed limit to complete current block group | | **to_block** | User-specified ending block (exclusive) | Never exceeds this limit | | **max*num*\*** | User-specified maximum number of results by type | May slightly exceed limit to complete current block group | ### Execution Steps 1. Server receives query and identifies the starting block group 2. It scans each block group, applying selection criteria 3. It joins related data according to the specified join mode 4. When a limit is reached, it finishes processing the current block group 5. It returns results with pagination information ### Understanding Pagination HyperSync uses a time-based pagination model that differs from traditional RPC calls: - By default, HyperSync has a **5-second query execution limit** - Within this time window, it processes as many blocks as possible - For example, starting with `from_block: 0` might progress to block 10 million in a single request - Each response includes a `next_block` value indicating where to resume for the next query - This differs from RPC calls where you typically specify fixed block ranges (e.g., 0-1000) #### Understanding nextBlock `nextBlock` is the block number immediately after the last block included in the response. Use it as the `fromBlock` of your next query if you want to continue scanning. Resuming from `nextBlock` gives you a continuous, non-overlapping scanβ€”no gaps, no duplicates. **Usage pattern:** Call `get` or `getEvents`, process the page, then if `nextBlock` is less than your desired end (`toBlock` or `archiveHeight`), set `fromBlock = nextBlock` and repeat: ```javascript let query = { fromBlock: 0, logs: [...], fieldSelection: {...} }; while (true) { const res = await client.get(query); // Process res.data... const targetEnd = query.toBlock ?? res.archiveHeight; if (res.nextBlock >= targetEnd) break; query = { ...query, fromBlock: res.nextBlock }; } ``` For most use cases, the `stream` function handles pagination automatically, making it the recommended approach for processing large ranges of blocks. ### Reverse Search HyperSync supports searching from the head of the chain backwards, which is useful for: - Block explorers showing the most recent activity - UIs displaying latest transactions for a user - Any use case where recent data is more relevant To use reverse search, add the `reverse: true` parameter to your stream call: ```javascript // Example of reverse search to get recent transactions const receiver = await client.stream(query, { reverse: true }); let count = 0; while (true) { let res = await receiver.recv(); if (res === null) { break; } for (const tx of res.data.transactions) { console.log(JSON.stringify(tx, null, 2)); } count += res.data.transactions.length; if (count >= 20) { break; } } ``` With reverse search, HyperSync starts from the latest block and works backwards, allowing you to efficiently access the most recent blockchain data first. ## Query Structure Reference A complete HyperSync query can include the following components: ### Core Query Parameters ```rust struct Query { /// The block to start the query from from_block: u64, /// The block to end the query at (exclusive) /// If not specified, the query runs until the end of available data to_block: Optional, /// Log selection criteria (OR relationship between selections) logs: Array, /// Transaction selection criteria (OR relationship between selections) transactions: Array, /// Trace selection criteria (OR relationship between selections) traces: Array, /// Block selection criteria (OR relationship between selections) blocks: Array, /// Whether to include all blocks in the requested range /// Default: only return blocks related to matched transactions/logs include_all_blocks: bool, /// Fields to include in the response field_selection: FieldSelection, /// Maximum results limits (approximate) max_num_blocks: Optional, max_num_transactions: Optional, max_num_logs: Optional, max_num_traces: Optional, /// Data relationship model (Default, JoinAll, or JoinNothing) join_mode: JoinMode, } ``` ### Selection Types #### Log Selection ```rust struct LogSelection { /// Contract addresses to match (empty = match all) address: Array
, /// Topics to match by position (empty = match all) /// Each array element corresponds to a topic position (0-3) /// Within each position, any matching value will satisfy the condition topics: Array>, } ``` #### Transaction Selection ```rust struct TransactionSelection { /// Sender addresses (empty = match all) /// Has AND relationship with 'to' field from: Array
, /// Recipient addresses (empty = match all) /// Has AND relationship with 'from' field to: Array
, /// Method signatures to match (first 4 bytes of input) sighash: Array, /// Transaction status to match (1 = success, 0 = failure) status: Optional, /// Transaction types to match (e.g., 0 = legacy, 2 = EIP-1559) type: Array, /// Created contract addresses to match contract_address: Array
, } ``` #### Block Selection ```rust struct BlockSelection { /// Block hashes to match (empty = match all) hash: Array, /// Miner/validator addresses to match (empty = match all) miner: Array
, } ``` #### Trace Selection ```rust struct TraceSelection { /// Sender addresses (empty = match all) /// Has AND relationship with 'to' field from: Array
, /// Recipient addresses (empty = match all) /// Has AND relationship with 'from' field to: Array
, /// Created contract addresses to match address: Array
, /// Call types to match (e.g., "call", "delegatecall") call_type: Array, /// Reward types to match (e.g., "block", "uncle") reward_type: Array, /// Trace types to match (e.g., "call", "create", "suicide", "reward") /// Python client: pass this as kind= type: Array, /// Method signatures to match (first 4 bytes of input) sighash: Array, } ``` #### Field Selection ```rust struct FieldSelection { /// Block fields to include in response block: Array, /// Transaction fields to include in response transaction: Array, /// Log fields to include in response log: Array, /// Trace fields to include in response trace: Array, } ``` ## Data Schema HyperSync organizes blockchain data into four main tables. Below are the available fields for each table. :::info Field Naming When specifying fields in your query, always use snake_case names (e.g., `block_number`, not `blockNumber`). ::: ### Block Fields ```python class BlockField(StrEnum): # Fields present on all EVM chains NUMBER = 'number' # Block number HASH = 'hash' # Block hash PARENT_HASH = 'parent_hash' # Parent block hash SHA3_UNCLES = 'sha3_uncles' # SHA3 of uncles data LOGS_BLOOM = 'logs_bloom' # Bloom filter for logs TRANSACTIONS_ROOT = 'transactions_root' # Root of transaction trie STATE_ROOT = 'state_root' # Root of state trie RECEIPTS_ROOT = 'receipts_root' # Root of receipts trie MINER = 'miner' # Miner/validator address EXTRA_DATA = 'extra_data' # Extra data field SIZE = 'size' # Block size in bytes GAS_LIMIT = 'gas_limit' # Block gas limit GAS_USED = 'gas_used' # Total gas used in block TIMESTAMP = 'timestamp' # Block timestamp (Unix time) # Optional fields β€” not present on all EVM chains (may be null) NONCE = 'nonce' # Block nonce (absent on some L2s) DIFFICULTY = 'difficulty' # Block difficulty (PoW chains only) TOTAL_DIFFICULTY = 'total_difficulty' # Total chain difficulty (PoW chains only) UNCLES = 'uncles' # Uncle block hashes (absent on some L2s) MIX_HASH = 'mix_hash' # Mix hash (absent on some L2s) BASE_FEE_PER_GAS = 'base_fee_per_gas' # EIP-1559 base fee (post-London chains only) BLOB_GAS_USED = 'blob_gas_used' # Total blob gas used (EIP-4844 chains only) EXCESS_BLOB_GAS = 'excess_blob_gas' # Excess blob gas (EIP-4844 chains only) PARENT_BEACON_BLOCK_ROOT = 'parent_beacon_block_root' # Parent beacon block root (EIP-4844 chains only) WITHDRAWALS_ROOT = 'withdrawals_root' # Root of withdrawals trie (post-Shanghai chains only) WITHDRAWALS = 'withdrawals' # Validator withdrawals (post-Shanghai chains only) L1_BLOCK_NUMBER = 'l1_block_number' # L1 block number (Arbitrum only) SEND_COUNT = 'send_count' # Send count (Arbitrum only) SEND_ROOT = 'send_root' # Send root (Arbitrum only) ``` ### Transaction Fields ```python class TransactionField(StrEnum): # Block-related fields BLOCK_HASH = 'block_hash' # The Keccak 256-bit hash of the block BLOCK_NUMBER = 'block_number' # Block number containing the transaction # Transaction identifiers HASH = 'hash' # Transaction hash (keccak hash of RLP encoded signed transaction) TRANSACTION_INDEX = 'transaction_index' # Index of the transaction in the block # Transaction participants FROM = 'from' # 160-bit address of the sender TO = 'to' # 160-bit address of the recipient (null for contract creation) # Gas information GAS = 'gas' # Gas limit set by sender GAS_PRICE = 'gas_price' # Wei paid per unit of gas GAS_USED = 'gas_used' # Actual gas used by the transaction CUMULATIVE_GAS_USED = 'cumulative_gas_used' # Total gas used in the block up to this transaction EFFECTIVE_GAS_PRICE = 'effective_gas_price' # Sum of base fee and tip paid per unit of gas # EIP-1559 fields MAX_PRIORITY_FEE_PER_GAS = 'max_priority_fee_per_gas' # Max priority fee (a.k.a. GasTipCap) MAX_FEE_PER_GAS = 'max_fee_per_gas' # Max fee per gas (a.k.a. GasFeeCap) # Transaction data INPUT = 'input' # Transaction input data or contract initialization code VALUE = 'value' # Amount of ETH transferred in wei NONCE = 'nonce' # Number of transactions sent by the sender # Signature fields V = 'v' # Replay protection value (based on chain_id) R = 'r' # The R field of the signature S = 's' # The S field of the signature Y_PARITY = 'y_parity' # Signature Y parity CHAIN_ID = 'chain_id' # Chain ID for replay protection (EIP-155) # Contract-related fields CONTRACT_ADDRESS = 'contract_address' # Address of created contract (for contract creation txs) # Transaction result fields STATUS = 'status' # Success (1) or failure (0) LOGS_BLOOM = 'logs_bloom' # Bloom filter for logs produced by this transaction ROOT = 'root' # State root (pre-Byzantium) # EIP-2930 fields ACCESS_LIST = 'access_list' # List of addresses and storage keys to pre-warm # EIP-4844 (blob transactions) fields MAX_FEE_PER_BLOB_GAS = 'max_fee_per_blob_gas' # Max fee per data gas (blob fee cap) BLOB_VERSIONED_HASHES = 'blob_versioned_hashes' # List of blob versioned hashes # Transaction type KIND = 'type' # Transaction type (0=legacy, 1=EIP-2930, 2=EIP-1559, 3=EIP-4844, 4=EIP-7702) # note - in old versions of the clients this was called 'kind', in newer versions its called 'type' # L2-specific fields (for rollups) L1_FEE = 'l1_fee' # Fee for L1 data (L1GasPrice Γ— L1GasUsed) L1_GAS_PRICE = 'l1_gas_price' # Gas price on L1 L1_GAS_USED = 'l1_gas_used' # Amount of gas consumed on L1 L1_FEE_SCALAR = 'l1_fee_scalar' # Multiplier for L1 fee calculation GAS_USED_FOR_L1 = 'gas_used_for_l1' # Gas spent on L1 calldata in L2 gas units ``` ### Log Fields ```python class LogField(StrEnum): # Log identification LOG_INDEX = 'log_index' # Index of the log in the block TRANSACTION_INDEX = 'transaction_index' # Index of the transaction in the block # Transaction information TRANSACTION_HASH = 'transaction_hash' # Hash of the transaction that created this log # Block information BLOCK_HASH = 'block_hash' # Hash of the block containing this log BLOCK_NUMBER = 'block_number' # Block number containing this log # Log content ADDRESS = 'address' # Contract address that emitted the event DATA = 'data' # Non-indexed data from the event # Topics (indexed parameters) TOPIC0 = 'topic0' # Event signature hash TOPIC1 = 'topic1' # First indexed parameter TOPIC2 = 'topic2' # Second indexed parameter TOPIC3 = 'topic3' # Third indexed parameter # Reorg information REMOVED = 'removed' # True if log was removed due to chain reorganization ``` ### Trace Fields ```python class TraceField(StrEnum): # Trace identification TRANSACTION_HASH = 'transaction_hash' # Hash of the transaction TRANSACTION_POSITION = 'transaction_position' # Index of the transaction in the block SUBTRACES = 'subtraces' # Number of sub-traces created during execution TRACE_ADDRESS = 'trace_address' # Array indicating position in the trace tree # Block information BLOCK_HASH = 'block_hash' # Hash of the block containing this trace BLOCK_NUMBER = 'block_number' # Block number containing this trace # Transaction participants FROM = 'from' # Address of the sender TO = 'to' # Address of the recipient (null for contract creation) # Value and gas VALUE = 'value' # ETH value transferred (in wei) GAS = 'gas' # Gas limit GAS_USED = 'gas_used' # Gas actually used # Call data INPUT = 'input' # Call data for function calls INIT = 'init' # Initialization code for contract creation OUTPUT = 'output' # Return data from the call # Contract information ADDRESS = 'address' # Contract address (for creation/destruction) CODE = 'code' # Contract code # Trace types and categorization TYPE = 'type' # Trace type (call, create, suicide, reward) CALL_TYPE = 'call_type' # Call type (call, delegatecall, staticcall, etc.) REWARD_TYPE = 'reward_type' # Reward type (block, uncle) # Other actors AUTHOR = 'author' # Address of receiver for reward transactions # Result information ERROR = 'error' # Error message if failed ``` For a complete list of all available fields, refer to the [HyperSync API Reference](https://docs.envio.dev/docs/HyperSync/hypersync-query). ## Response Structure When you execute a HyperSync query, the response includes both metadata and the requested data: ```rust struct QueryResponse { /// Current height of the blockchain in HyperSync archive_height: Optional, /// Block number immediately after the last block included in this response. /// Use as from_block in your next query for pagination. next_block: u64, /// Query execution time in milliseconds total_execution_time: u64, /// The actual blockchain data matching your query data: ResponseData, /// Information to help handle chain reorganizations rollback_guard: Optional, } ``` The `next_block` value tells you where to resume scanning. See [Understanding nextBlock](#understanding-nextblock) for a clear definition and usage pattern. ### Rollback Guard The optional `rollback_guard` lets you detect chain reorganizations (reorgs) between successive queries, so you can re-fetch any data that has become stale. ```rust struct RollbackGuard { /// Last block scanned in this query block_number: u64, /// Timestamp of the last block scanned timestamp: i64, /// Hash of the last block scanned hash: Hash, /// First block scanned in this query first_block_number: u64, /// Parent hash of the first block scanned first_parent_hash: Hash, } ``` The guard is `Optional`: it is present whenever the response covers blocks near the chain tip (where reorgs can still happen) and absent for queries that return no data. #### How HyperSync handles reorgs internally As HyperSync ingests new blocks it checks each block's `parent_hash` against the previous block's `hash`. When a mismatch is detected, HyperSync re-syncs the affected blocks and continues serving the canonical chain. A single query response is always internally consistent: you will never receive a mix of blocks from different forks. The rollback guard exists to detect reorgs that happen _between_ successive queries, where data you fetched earlier may now be stale. #### Detecting a reorg After each query, store the guard's `block_number` and `hash`. On the next query, compare: - `previous response.hash` (last block you saw) - `next response.first_parent_hash` (parent of the first block in the new batch) If they match, the chain is intact. If they differ, a reorg occurred somewhere between the two queries. ``` Query N: rollback_guard.hash = 0xABC... (stored) Query N+1: rollback_guard.first_parent_hash = 0xABC... match -> no reorg = 0xDEF... mismatch -> reorg ``` #### Recovering from a reorg The guard tells you _that_ a reorg happened but not how deep. To find the depth, keep enough history to cover your chain's reorg threshold (for example, 200 blocks for Polygon) and walk backwards: re-fetch each stored block's hash and compare. The first block whose hash still matches is the last canonical block; rewind your downstream state to there and resume querying. ```python async def get_block_hash(client, block_number): res = await client.get(hypersync.Query( from_block=block_number, to_block=block_number + 1, include_all_blocks=True, field_selection=hypersync.FieldSelection(block=[hypersync.BlockField.HASH]), )) return res.data.blocks[0].hash history = [] # list of (block_number, hash) while True: res = await client.get(query) guard = res.rollback_guard if guard is None: process(res.data) query.from_block = res.next_block continue if history and guard.first_parent_hash != history[-1][1]: # Walk back to find the last block still on chain. while history: block_num, stored_hash = history[-1] if await get_block_hash(client, block_num) == stored_hash: break history.pop() rewind_to = history[-1][0] + 1 if history else query.from_block rollback_state_to(rewind_to) query.from_block = rewind_to continue process(res.data) history.append((guard.block_number, guard.hash)) cutoff = guard.block_number - REORG_THRESHOLD history = [(b, h) for b, h in history if b >= cutoff] query.from_block = res.next_block ``` :::tip Consider HyperIndex [HyperIndex](https://docs.envio.dev/docs/HyperIndex/overview) handles all of this for you: it tracks recent block hashes, locates the reorg point, and rolls back database state automatically. See [Reorgs Support](https://docs.envio.dev/docs/HyperIndex/reorgs-support) for details. ::: ## Stream and Collect Functions For continuous data processing or building data pipelines, client libraries provide `stream` and `collect` functions that wrap the base query functionality. :::caution Tip of Chain Warning These functions are not designed for use at the blockchain tip where rollbacks may occur. For real-time data near the chain tip, implement a custom loop using the `get` functions and handle rollbacks manually. ::: ### Stream Function The `stream` function: - Runs multiple queries concurrently - Returns a stream handle that yields results as they're available - Optimizes performance through pipelined decoding/decompression - Continues until reaching either `to_block` or the chain height at stream start ### Collect Functions The `collect` functions: - Call `stream` internally and aggregate results - Offer different output formats (JSON, Parquet) - Handle data that may not fit in memory :::warning Resource Management Always call `close()` on stream handles when finished to prevent resource leaks, especially if creating multiple streams. ::: ## Working with Join Modes HyperSync "joins" connect related blockchain data automatically. Unlike SQL joins that combine rows from different tables, HyperSync joins determine which related records to include in the response. ### Default Join Mode (logs β†’ transactions β†’ traces β†’ blocks) With the default join mode: 1. When you query logs, you automatically get their associated transactions 2. Those transactions' traces are also included 3. The blocks containing these transactions are included ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β” β”‚ Logs β”‚ ──> β”‚ Transactions β”‚ ──> β”‚ Tracesβ”‚ ──> β”‚ Blocksβ”‚ β””β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### JoinAll Mode JoinAll creates a more comprehensive network of related data: ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β–Ό β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β” <──> β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” <──> β”Œβ”€β”€β”€β”€β”€β”€β”€β” <──> β”Œβ”€β”€β”€β”€β”€β”€β”€β” β”‚ Logs β”‚ β”‚ Transactions β”‚ β”‚ Tracesβ”‚ β”‚ Blocksβ”‚ β””β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”˜ ``` For example, if you query a trace: 1. You get the transaction that created it 2. You get ALL logs from that transaction (not just the ones matching your criteria) 3. You get ALL traces from that transaction 4. You get the block containing the transaction ### JoinNothing Mode JoinNothing is the most restrictive: ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β” β”‚ Logs β”‚ β”‚ Transactions β”‚ β”‚ Tracesβ”‚ β”‚ Blocksβ”‚ β””β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”˜ ``` Only data directly matching your selection criteria is returned, with no related records included. ## Best Practices To get the most out of HyperSync queries: 1. **Minimize field selection** - Only request fields you actually need to improve performance 2. **Use appropriate limits** - Set `max_num_*` parameters to control response size 3. **Choose the right join mode** - Use `JoinNothing` for minimal data, `JoinAll` for complete context 4. **Process in chunks** - For large datasets, use pagination or the `stream` function 5. **Consider Parquet** - For analytical workloads, use `collect_parquet` for efficient storage 6. **Handle chain tip carefully** - Near the chain tip, implement custom rollback handling --- # HyperSync Preset Queries > Explore ready-to-use HyperSync query presets to fetch blocks, transactions, and logs efficiently. HyperSync's client libraries include helper functions that build common queries. These presets are useful when you need raw blockchain objects without crafting a query manually. Each preset returns a `Query` object so you can pass it directly to `client.get`, `client.stream`, or `client.collect`. The names below are the Python ones. Node.js and TypeScript expose the same four presets in camelCase, and Rust uses a different naming scheme again. See [Other client libraries](#other-client-libraries). ## Available Presets ### `preset_query_blocks_and_transactions(from_block, to_block=None)` Returns every block and all associated transactions within the supplied block range. Leave `to_block` unset to query up to the current chain head. ```python import hypersync import asyncio async def main(): client = hypersync.HypersyncClient( hypersync.ClientConfig( url="https://eth.hypersync.xyz", api_token="your-token-here", # Get from https://docs.envio.dev/docs/HyperSync/api-tokens ) ) query = hypersync.preset_query_blocks_and_transactions(17_000_000, 17_000_050) result = await client.get(query) print(f"Query returned {len(result.data.blocks)} blocks and {len(result.data.transactions)} transactions") asyncio.run(main()) ``` ### `preset_query_blocks_and_transaction_hashes(from_block, to_block=None)` Returns each block in the range along with only the transaction hashes. ### `preset_query_logs(address, from_block, to_block=None)` Fetches all logs emitted by a single contract address in the given block range. The address is the first argument. ```python logs_res = await client.get( hypersync.preset_query_logs("0xdAC17F958D2ee523a2206206994597C13D831ec7", 17_000_000, 17_000_050) ) ``` ### `preset_query_logs_of_event(address, topic0, from_block, to_block=None)` Fetches logs emitted by a contract address that match a single event topic. `topic0` is the hashed event signature, not the human-readable one. ```python # keccak256("Transfer(address,address,uint256)") transfer_topic0 = "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef" logs_res = await client.get( hypersync.preset_query_logs_of_event( "0xdAC17F958D2ee523a2206206994597C13D831ec7", transfer_topic0, 17_000_000, 17_000_050, ) ) ``` ## Other client libraries ### Node.js and TypeScript `@envio-dev/hypersync-client` exports the same four presets in camelCase, with the same argument order. The Python snake_case names are not defined in the Node client. ```typescript import { presetQueryBlocksAndTransactions, presetQueryBlocksAndTransactionHashes, presetQueryLogs, presetQueryLogsOfEvent, } from "@envio-dev/hypersync-client"; const query = presetQueryLogs( "0xdAC17F958D2ee523a2206206994597C13D831ec7", 17_000_000, 17_000_050 ); ``` ### Rust The Rust client groups these in a `preset_query` module, drops the `preset_query_` prefix, and takes the block range first rather than last. ```rust use hypersync_client::preset_query; let query = preset_query::logs(17_000_000, Some(17_000_050), contract_address); ``` The module provides `blocks_and_transactions`, `blocks_and_transaction_hashes`, `logs`, `logs_of_event`, `transactions`, and `transactions_from_address`. The last two have no Python or Node.js equivalent. For runnable examples, see the [Python](https://github.com/enviodev/hypersync-client-python) and [Node.js](https://github.com/enviodev/hypersync-client-node) client repositories. Use these helpers whenever you need a quick query without specifying field selections or joins manually. --- # Using curl with HyperSync > Explore how to quickly fetch blockchain data using curl commands with HyperSync. This guide demonstrates how to interact with HyperSync using direct HTTP requests via curl. These examples provide a quick way to explore HyperSync functionality without installing client libraries. :::info Recommended Approach We highly recommend trying these curl examples as they're super quick and easy to run directly in your terminal. It's one of the fastest ways to experience HyperSync's performance firsthand and see just how quickly you can retrieve blockchain data without any setup overhead. Simply copy, paste, and be amazed by the response speed! While curl requests are technically slower than our client libraries (since they use HTTP rather than binary data transfer protocols), they're still impressively fast and provide an excellent demonstration of HyperSync's capabilities without any installation requirements. ::: ## Table of Contents - [Curl vs. Client Libraries](#curl-vs-client-libraries) - [When to Use curl (JSON API)](#when-to-use-curl-json-api) - [When to Use Client Libraries](#when-to-use-client-libraries) - [Common Use Cases](#common-use-cases) - [Get All ERC-20 Transfers for an Address](#get-all-erc-20-transfers-for-an-address) - [Get All Logs for a Smart Contract](#get-all-logs-for-a-smart-contract) - [Get Blob Data for the Optimism Chain](#get-blob-data-for-the-optimism-chain) - [Get Mint USDC Events](#get-mint-usdc-events) - [Get All Transactions for an Address](#get-all-transactions-for-an-address) - [Get Successful or Failed Transactions](#get-successful-or-failed-transactions) ## Curl vs. Client Libraries When deciding whether to use curl commands or client libraries, consider the following comparison: ### When to Use curl (JSON API) - **Quick Prototyping**: Test endpoints and explore data structure without setup - **Simple Scripts**: Perfect for shell scripts and automation - **Language Independence**: When working with languages without HyperSync client libraries - **API Exploration**: When learning the HyperSync API capabilities ### When to Use Client Libraries - **Production Applications**: For stable, maintained codebases - **Complex Data Processing**: When working with large datasets or complex workflows - **Performance**: Client libraries offer automatic compression and pagination - **Error Handling**: Built-in retry mechanisms and better error reporting - **Data Formats**: Support for efficient formats like Apache Arrow ## Common Use Cases ### Get All ERC-20 Transfers for an Address This example filters for all ERC-20 transfer events involving a specific address, either as sender or recipient. Feel free to swap your address into the example. **What this query does:** - Filters logs for the Transfer event signature (topic0) - Matches when the address appears in either topic1 (sender) or topic2 (recipient) - Also includes direct transactions to/from the address ```bash curl --request POST \ --url https://eth.hypersync.xyz/query \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "from_block": 0, "logs": [ { "topics": [ [ "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef" ], [], [ "0x0000000000000000000000001e037f97d730Cc881e77F01E409D828b0bb14de0" ] ] }, { "topics": [ [ "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef" ], [ "0x0000000000000000000000001e037f97d730Cc881e77F01E409D828b0bb14de0" ], [] ] } ], "transactions": [ { "from": [ "0x1e037f97d730Cc881e77F01E409D828b0bb14de0" ] }, { "to": [ "0x1e037f97d730Cc881e77F01E409D828b0bb14de0" ] } ], "field_selection": { "block": [ "number", "timestamp", "hash" ], "log": [ "block_number", "log_index", "transaction_index", "data", "address", "topic0", "topic1", "topic2", "topic3" ], "transaction": [ "block_number", "transaction_index", "hash", "from", "to", "value", "input" ] } }' ``` ### Get All Logs for a Smart Contract This example retrieves all event logs emitted by a specific contract (USDC in this case). **Key points:** - Sets `from_block: 0` to scan from the beginning of the chain - Uses `next_block` in the response for pagination to fetch subsequent data - Includes relevant block, log, and transaction fields ```bash curl --request POST \ --url https://eth.hypersync.xyz/query \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "from_block": 0, "logs": [ { "address": ["0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48"] } ], "field_selection": { "block": [ "number", "timestamp", "hash" ], "log": [ "block_number", "log_index", "transaction_index", "data", "address", "topic0", "topic1", "topic2", "topic3" ], "transaction": [ "block_number", "transaction_index", "hash", "from", "to", "value", "input" ] } }' ``` ### Get Blob Data for the Optimism Chain This example finds blob transactions used by the Optimism chain for data availability. **Key points:** - Starts at a relatively recent block (20,000,000) - Filters for transactions from the Optimism sequencer address - Specifically looks for type 3 (blob) transactions - Results can be used to retrieve the actual blob data from Ethereum ```bash curl --request POST \ --url https://eth.hypersync.xyz/query \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "from_block": 20000000, "transactions": [ { "from": ["0x6887246668a3b87F54DeB3b94Ba47a6f63F32985"], "to": ["0xFF00000000000000000000000000000000000010"], "type": [3] } ], "field_selection": { "block": [ "number", "timestamp", "hash" ], "transaction": [ "block_number", "transaction_index", "hash", "from", "to", "type" ] } }' ``` ### Get Mint USDC Events This example identifies USDC token minting events. **How it works:** - Filters for the USDC contract address - Looks for Transfer events (topic0) - Specifically matches when topic1 (from address) is the zero address, indicating a mint - Returns detailed information about each mint event ```bash curl --request POST \ --url https://eth.hypersync.xyz/query \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "from_block": 0, "logs": [ { "address": ["0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48"], "topics": [ [ "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef" ], [ "0x0000000000000000000000000000000000000000000000000000000000000000" ], [] ] } ], "field_selection": { "block": [ "number", "timestamp", "hash" ], "log": [ "block_number", "log_index", "transaction_index", "data", "address", "topic0", "topic1", "topic2", "topic3" ], "transaction": [ "block_number", "transaction_index", "hash", "from", "to", "value", "input" ] } }' ``` ### Get All Transactions for an Address This example retrieves all transactions where a specific address is either the sender or receiver. **Implementation notes:** - Starts from a specific block (15,362,000) for efficiency - Uses two transaction filters in an OR relationship - Only includes essential fields in the response - Multiple queries may be needed for complete history ```bash curl --request POST \ --url https://eth.hypersync.xyz/query \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "from_block": 15362000, "transactions": [ { "from": ["0xdb255746609baadd67ef44fc15b5e1d04befbca7"] }, { "to": ["0xdb255746609baadd67ef44fc15b5e1d04befbca7"] } ], "field_selection": { "block": [ "number", "timestamp", "hash" ], "transaction": [ "block_number", "transaction_index", "hash", "from", "to" ] } }' ``` ### Get Successful or Failed Transactions This example shows how to filter transactions based on their status (successful or failed) for recent blocks. **How it works:** 1. First, query the current chain height 2. Calculate a starting point (current height minus 10) 3. Query transactions with status=1 (successful) or status=0 (failed) ```bash # Get current height and calculate starting block height=$((`curl https://eth.hypersync.xyz/height | jq .height` - 10)) # Query successful transactions (change status to 0 for failed transactions) curl --request POST \ --url https://eth.hypersync.xyz/query \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data "{ \"from_block\": ${height}, \"transactions\": [ { \"status\": 1 } ], \"field_selection\": { \"block\": [ \"number\", \"timestamp\", \"hash\" ], \"transaction\": [ \"block_number\", \"transaction_index\", \"hash\", \"from\", \"to\" ] } }" ``` --- # Stream Config & Tuning > Configure and tune the HyperSync streaming engine (concurrency, response size targets, batch sizes) and find the best config for your query with the built-in observer. When you `stream` (or `collect`) with a HyperSync client, the engine fans your block range out across many concurrent HTTP requests, sizes each request automatically, and delivers results to you **in block order**. `StreamConfig` controls that engine. :::tip TL;DR **The defaults are good for most workloads, so you usually don't need to touch this.** Reach for tuning when you want to squeeze more throughput out of a specific workload, or to bound memory. The fastest way to find a good config for *your* query is the [`tune_stream`](#find-the-best-config-for-your-query) tool, which sweeps configs against your query and prints a comparison table. ::: ## Quick decision guide | Your situation | Do this | | --- | --- | | Just getting started | Use the defaults. | | Want maximum throughput on a busy contract (lots of logs) | Raise `concurrency` (for example 20). | | Scanning a wide range for a rare event | Keep `concurrency` moderate (around 10); a larger `batch_size` helps. | | Pulling full blocks + all transactions | Defaults are fine; the adaptive buffer handles the large responses. | | Memory constrained | Set `max_buffered_bytes` to a fixed cap. | | Hitting API rate limits | The client waits out limits and retries automatically. To read the quota yourself, see [Inspecting rate limits](#inspecting-rate-limits-from-your-code); for higher limits, [upgrade your plan](https://docs.envio.dev/docs/HyperSync/api-tokens.md). | | Not sure | Run [`tune_stream`](#find-the-best-config-for-your-query) against your query. | ## How the engine works (the 30-second model) Two ideas make every knob obvious: 1. **One HTTP request is one unit of work.** The engine sizes each request to land near `response_bytes_target` bytes, based on the byte-density it has measured so far. If the server returns less than requested (a "truncation"), the leftover range becomes a gap that any free worker backfills in parallel. 2. **Delivery is in block order.** Results are buffered and handed to you contiguously, so the stream yields one response per HTTP response in ascending (or `reverse`) block order. So `concurrency` sets how many requests run in parallel, and `response_bytes_target` sets how big each response is. Because truncations are backfilled automatically, the engine is forgiving: an over-estimated request size corrects itself rather than failing. ## Configuration options Field names below are shown in `snake_case` (Rust, Python). **Node and TypeScript use `camelCase`** (for example `response_bytes_target` becomes `responseBytesTarget`). | Option | Default | What it does | | --- | --- | --- | | `concurrency` | `10` | Number of requests in flight, and your main throughput knob. `0` is an error, `1` streams sequentially, `2` or more uses the parallel scheduler. | | `response_bytes_target` | `400_000` | Target size in bytes for each response. Each request is sized to aim here. Raise it for fewer, larger responses. | | `batch_size` | `1_000` | Initial block range for the first wave of requests, before any density has been measured. Also the fallback. | | `min_batch_size` | `200` | Lower clamp on the projected block count, to avoid tiny ranges. | | `max_batch_size` | _unset_ | Optional hard cap on blocks per request. Unset means no cap (over-shoot self-corrects via backfill). Set it only if you specifically want to bound blocks per request. | | `max_buffered_bytes` | _unset_ | Cap on bytes of fetched-but-undelivered data held for re-ordering (consumer backpressure). Unset means adaptive (it grows with the largest response seen) so byte-heavy pulls stay pipelined. Set a fixed value to bound memory. | | `reverse` | `false` | Stream from the top of the range downward. | | `max_num_blocks`, `max_num_transactions`, `max_num_logs`, `max_num_traces` | _unset_ | Stop the stream once this many of an entity have been delivered. | | `column_mapping`, `event_signature`, `hex_output` | none | Output shaping (decoding, hex formatting). Not performance knobs. | :::info Breaking change in v1.3.0 / streaming v2 `response_bytes_floor` and `response_bytes_ceiling` were replaced by a single `response_bytes_target`. `max_batch_size` became optional (unset means no cap). `max_buffered_bytes` was added. If you set the old fields, switch to `response_bytes_target`. ::: ## Tuning recipes These are good starting points. Confirm against your own query with [`tune_stream`](#find-the-best-config-for-your-query). ### Dense: busy contracts and all-logs Lots of matching data per block. Throughput scales with parallelism. ```python # Python config = hypersync.StreamConfig(concurrency=20, response_bytes_target=400_000) ``` ```typescript // Node / TypeScript const config = { concurrency: 20, responseBytesTarget: 400_000 }; ``` ```rust // Rust has a ready-made preset let config = StreamConfig::dense(); ``` ### Sparse: rare events over a wide range Most blocks match nothing. Raising concurrency past the default tends to just fragment the empty range into more, smaller requests without adding throughput, so keep it moderate. A larger `batch_size` lets the first wave cover more ground. ```python config = hypersync.StreamConfig(concurrency=10, batch_size=20_000) ``` ```typescript const config = { concurrency: 10, batchSize: 20_000 }; ``` ```rust let config = StreamConfig::sparse(); ``` ### Archival: full blocks and all transactions Each response is many megabytes, so the run is bound by the re-order buffer rather than concurrency. Leave `max_buffered_bytes` unset so the adaptive buffer keeps the pipeline full. ```python config = hypersync.StreamConfig(concurrency=12) ``` ```typescript const config = { concurrency: 12 }; ``` ```rust let config = StreamConfig::archival(); ``` ## Find the best config for your query Rather than guess, measure. The Rust client ships a standalone **`tune_stream`** example that runs your query under a grid of configs and prints a comparison table: throughput, request count, truncation rate, and how close responses land to the target. It takes a query as JSON, so it works for any query regardless of which client language you use. ### 1. Save your query as JSON ```json title="query.json" { "from_block": 18000000, "to_block": 18100000, "logs": [ { "topics": [["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]] } ], "field_selection": { "log": ["block_number", "log_index", "data", "topic0"] } } ``` ### 2. Run the sweep ```bash git clone https://github.com/enviodev/hypersync-client-rust cd hypersync-client-rust export ENVIO_API_TOKEN= export CHAIN_ID=1 # 1 = Ethereum mainnet # Sweep a grid of configs and print a comparison table cargo run -p tune_stream -- query.json # Or a single detailed run with the default config cargo run -p tune_stream -- query.json --single ``` The table shows, per config: requests, truncation %, blocks/s, MB/s, the mean size-vs-target ratio, and the observed buffer and in-flight counts. Pick the config with the best throughput for your workload. ### 3. Apply the winning config Copy `response_bytes_target`, `concurrency`, and any other fields from the best row into your client's `StreamConfig`. ### Rust: attach an observer in your own code Rust users can read the same metrics live, without the example, via the observer API: ```rust use std::sync::Arc; use hypersync_client::{Client, StreamConfig, StreamMetrics, StreamObserver}; let metrics = Arc::new(StreamMetrics::new()); let observer: Arc = metrics.clone(); let mut rx = client .stream_arrow_with_observer(query, StreamConfig::default(), observer) .await?; while let Some(res) = rx.recv().await { let _ = res?; // consume the stream } let summary = metrics.summary(); println!( "requests={} truncated={:.0}% blocks/s={:.0} mean size ratio={:.2}", summary.num_requests, summary.truncation_rate * 100.0, summary.blocks_per_sec, summary.mean_size_ratio, ); ``` The plain `stream` and `stream_arrow` methods do no metrics work; `RequestStats` are only built when you attach an observer. (Surfacing this handle in the Node and Python clients is a fast-follow; until then, use `tune_stream` for those languages.) ## Reading the metrics When tuning, the two numbers worth watching are: - **Mean size ratio**, which is response size divided by `response_bytes_target`. Around `1.0` means responses are landing on target. Well below `1.0` means the server is returning smaller responses than your target, which is perfectly normal for selective queries or minimal field selections; raising `response_bytes_target` won't change it, and the way to go faster is usually more `concurrency`. - **Blocks per second**, which is the throughput you are tuning for. Truncation is shown too, but you can usually ignore it. Some truncation is normal and harmless because the leftover range is backfilled automatically, and a sparse query that selects only a few fields will often show small, frequently-truncated responses while still streaming quickly. It is only worth a look if throughput is poor. ## Rate limits The clients handle rate limits for you: when the server signals a limit (HTTP 429) the client waits for the window to reset and retries, so a stream slows down rather than failing. If you want more headroom or higher throughput, [upgrade your plan](https://docs.envio.dev/docs/HyperSync/api-tokens.md). If you need the client to make fewer requests per unit time (for example to fit a fixed request budget), `concurrency` is the lever. It trades throughput for request volume: fewer requests run in parallel, so fewer go out per unit time. Setting `concurrency = 1` streams sequentially, one request at a time. ### Inspecting rate limits from your code Sometimes waiting is not enough: you may want to show quota in a dashboard, or coordinate a budget across several processes that share one token. Every response carries the server's `x-ratelimit-*` headers, and the clients expose them under the same names. | Purpose | Rust | Node (EVM client) | | --- | --- | --- | | Query and get the headers back | `get_with_rate_limit`, `get_arrow_with_rate_limit` | `getWithRateLimit` | | Last observed headers, no request | `rate_limit_info()` | `rateLimitInfo()` | | Wait out an exhausted window | `wait_for_rate_limit()` | `waitForRateLimit()` | | Opt out of the automatic pre-request wait | `ClientConfig::proactive_rate_limit_sleep` | `proactiveRateLimitSleep` | The query methods return a `QueryResponseWithRateLimit`, which pairs the normal response with a `RateLimitInfo`: | Field | Header | Meaning | | --- | --- | --- | | `limit` | `x-ratelimit-limit` | Total budget for the current window. | | `remaining` | `x-ratelimit-remaining` | Budget left in the window. | | `reset_secs` / `resetSecs` | `x-ratelimit-reset` | Seconds until the window resets. | | `cost` | `x-ratelimit-cost` | Budget consumed per request. | Every field is optional, because parsing is best-effort: a missing or malformed header just leaves it unset. Note that `remaining` counts **budget units, not requests** - divide it by `cost` for the number of requests you have left (a limit of 50 with a cost of 10 is 5 requests per window). `proactive_rate_limit_sleep` is on by default. With it enabled the client checks the last observed headers before sending and waits out a window it knows is exhausted, instead of spending a request on a certain 429. Turn it off if you are doing your own scheduling. :::note One difference between the EVM and Solana clients The names and shapes are identical, but the 429 behavior is not. On the **EVM** client `get_with_rate_limit` does **not** retry a 429: it returns with no response body and the rate-limit info filled in, and retrying is your job. On the **Solana** client the `*_with_rate_limit` methods retry exactly like the plain `get` / `get_arrow`, so the response is always present. The plain `get` methods retry on both. ::: ## Full default reference ```text concurrency = 10 response_bytes_target = 400_000 batch_size = 1_000 min_batch_size = 200 max_batch_size = unset (no cap) max_buffered_bytes = unset (adaptive) reverse = false max_num_blocks / transactions / logs / traces = unset ``` --- # HyperSync Supported Networks > See all networks currently supported by HyperSync, including reliability notes and the criteria we use when adding new chains. :::note We are rapidly adding new supported networks. If you don't see your network here or would like us to add a network to HyperSync, pop us a message in our [Discord](https://discord.gg/envio). ::: :::info The Tier is the level of support (and therefore reliability) based on the infrastructure running the chain. We are actively working to make the tier distinctions more clear and transparent to our users. Currently, tiers relate to various service quality aspects including: - Allocated resources and compute power - Query processing speed - Infrastructure redundancy - Backup frequency and retention - Multi-region availability - Priority for upgrades and new features - SLA guarantees While detailed tier specifications are still being finalized, we're committed to providing transparent service level information in the near future. If you are a network operator or user and would like improved service support or to discuss upgrading a chain's level of support, please reach out to us in [Discord](https://discord.gg/envio). ::: | Network Name | Network ID | HyperSync URL | HyperRPC URL | | --- | --- | --- | --- | | Ab | 36888 | https://ab.hypersync.xyz or https://36888.hypersync.xyz | https://ab.rpc.hypersync.xyz or https://36888.rpc.hypersync.xyz | | Abstract | 2741 | https://abstract.hypersync.xyz or https://2741.hypersync.xyz | https://abstract.rpc.hypersync.xyz or https://2741.rpc.hypersync.xyz | | Arbitrum | 42161 | https://arbitrum.hypersync.xyz or https://42161.hypersync.xyz | https://arbitrum.rpc.hypersync.xyz or https://42161.rpc.hypersync.xyz | | Arbitrum Nova | 42170 | https://arbitrum-nova.hypersync.xyz or https://42170.hypersync.xyz | https://arbitrum-nova.rpc.hypersync.xyz or https://42170.rpc.hypersync.xyz | | Arbitrum Sepolia | 421614 | https://arbitrum-sepolia.hypersync.xyz or https://421614.hypersync.xyz | https://arbitrum-sepolia.rpc.hypersync.xyz or https://421614.rpc.hypersync.xyz | | Arc | 5042 | https://arc.hypersync.xyz or https://5042.hypersync.xyz | https://arc.rpc.hypersync.xyz or https://5042.rpc.hypersync.xyz | | Arc Testnet | 5042002 | https://arc-testnet.hypersync.xyz or https://5042002.hypersync.xyz | https://arc-testnet.rpc.hypersync.xyz or https://5042002.rpc.hypersync.xyz | | Aurora | 1313161554 | https://aurora.hypersync.xyz or https://1313161554.hypersync.xyz | https://aurora.rpc.hypersync.xyz or https://1313161554.rpc.hypersync.xyz | | Avalanche | 43114 | https://avalanche.hypersync.xyz or https://43114.hypersync.xyz | https://avalanche.rpc.hypersync.xyz or https://43114.rpc.hypersync.xyz | | Base | 8453 | https://base.hypersync.xyz or https://8453.hypersync.xyz | https://base.rpc.hypersync.xyz or https://8453.rpc.hypersync.xyz | | Base Sepolia | 84532 | https://base-sepolia.hypersync.xyz or https://84532.hypersync.xyz | https://base-sepolia.rpc.hypersync.xyz or https://84532.rpc.hypersync.xyz | | Base Traces* | 8453 | Access on request - [contact us](https://discord.gg/envio) | Access on request - [contact us](https://discord.gg/envio) | | Berachain | 80094 | https://berachain.hypersync.xyz or https://80094.hypersync.xyz | https://berachain.rpc.hypersync.xyz or https://80094.rpc.hypersync.xyz | | Blast | 81457 | https://blast.hypersync.xyz or https://81457.hypersync.xyz | https://blast.rpc.hypersync.xyz or https://81457.rpc.hypersync.xyz | | Boba | 288 | https://boba.hypersync.xyz or https://288.hypersync.xyz | https://boba.rpc.hypersync.xyz or https://288.rpc.hypersync.xyz | | Bsc | 56 | https://bsc.hypersync.xyz or https://56.hypersync.xyz | https://bsc.rpc.hypersync.xyz or https://56.rpc.hypersync.xyz | | Bsc Testnet | 97 | https://bsc-testnet.hypersync.xyz or https://97.hypersync.xyz | https://bsc-testnet.rpc.hypersync.xyz or https://97.rpc.hypersync.xyz | | Celo | 42220 | https://celo.hypersync.xyz or https://42220.hypersync.xyz | https://celo.rpc.hypersync.xyz or https://42220.rpc.hypersync.xyz | | Chiliz | 88888 | https://chiliz.hypersync.xyz or https://88888.hypersync.xyz | https://chiliz.rpc.hypersync.xyz or https://88888.rpc.hypersync.xyz | | Citrea | 4114 | https://citrea.hypersync.xyz or https://4114.hypersync.xyz | https://citrea.rpc.hypersync.xyz or https://4114.rpc.hypersync.xyz | | Citrea Testnet | 5115 | https://citrea-testnet.hypersync.xyz or https://5115.hypersync.xyz | https://citrea-testnet.rpc.hypersync.xyz or https://5115.rpc.hypersync.xyz | | Curtis | 33111 | https://curtis.hypersync.xyz or https://33111.hypersync.xyz | https://curtis.rpc.hypersync.xyz or https://33111.rpc.hypersync.xyz | | Cyber | 7560 | https://cyber.hypersync.xyz or https://7560.hypersync.xyz | https://cyber.rpc.hypersync.xyz or https://7560.rpc.hypersync.xyz | | Eth Traces* | 1 | Access on request - [contact us](https://discord.gg/envio) | Access on request - [contact us](https://discord.gg/envio) | | Ethereum Mainnet | 1 | https://eth.hypersync.xyz or https://1.hypersync.xyz | https://eth.rpc.hypersync.xyz or https://1.rpc.hypersync.xyz | | Etherlink | 42793 | https://etherlink.hypersync.xyz or https://42793.hypersync.xyz | https://etherlink.rpc.hypersync.xyz or https://42793.rpc.hypersync.xyz | | Fantom | 250 | https://fantom.hypersync.xyz or https://250.hypersync.xyz | https://fantom.rpc.hypersync.xyz or https://250.rpc.hypersync.xyz | | Flare | 14 | https://flare.hypersync.xyz or https://14.hypersync.xyz | https://flare.rpc.hypersync.xyz or https://14.rpc.hypersync.xyz | | Fraxtal | 252 | https://fraxtal.hypersync.xyz or https://252.hypersync.xyz | https://fraxtal.rpc.hypersync.xyz or https://252.rpc.hypersync.xyz | | Fuji | 43113 | https://fuji.hypersync.xyz or https://43113.hypersync.xyz | https://fuji.rpc.hypersync.xyz or https://43113.rpc.hypersync.xyz | | Gnosis | 100 | https://gnosis.hypersync.xyz or https://100.hypersync.xyz | https://gnosis.rpc.hypersync.xyz or https://100.rpc.hypersync.xyz | | Gnosis Chiado | 10200 | https://gnosis-chiado.hypersync.xyz or https://10200.hypersync.xyz | https://gnosis-chiado.rpc.hypersync.xyz or https://10200.rpc.hypersync.xyz | | Gnosis Traces | 100 | Access on request - [contact us](https://discord.gg/envio) | Access on request - [contact us](https://discord.gg/envio) | | Harmony Shard 0 | 1666600000 | https://harmony-shard-0.hypersync.xyz or https://1666600000.hypersync.xyz | https://harmony-shard-0.rpc.hypersync.xyz or https://1666600000.rpc.hypersync.xyz | | Hoodi | 560048 | https://hoodi.hypersync.xyz or https://560048.hypersync.xyz | https://hoodi.rpc.hypersync.xyz or https://560048.rpc.hypersync.xyz | | Hyperliquid | 999 | https://hyperliquid.hypersync.xyz or https://999.hypersync.xyz | https://hyperliquid.rpc.hypersync.xyz or https://999.rpc.hypersync.xyz | | Injective* | 1776 | https://injective.hypersync.xyz or https://1776.hypersync.xyz | https://injective.rpc.hypersync.xyz or https://1776.rpc.hypersync.xyz | | Ink | 57073 | https://ink.hypersync.xyz or https://57073.hypersync.xyz | https://ink.rpc.hypersync.xyz or https://57073.rpc.hypersync.xyz | | Katana | 747474 | https://katana.hypersync.xyz or https://747474.hypersync.xyz | https://katana.rpc.hypersync.xyz or https://747474.rpc.hypersync.xyz | | Linea | 59144 | https://linea.hypersync.xyz or https://59144.hypersync.xyz | https://linea.rpc.hypersync.xyz or https://59144.rpc.hypersync.xyz | | Lisk | 1135 | https://lisk.hypersync.xyz or https://1135.hypersync.xyz | https://lisk.rpc.hypersync.xyz or https://1135.rpc.hypersync.xyz | | Lukso | 42 | https://lukso.hypersync.xyz or https://42.hypersync.xyz | https://lukso.rpc.hypersync.xyz or https://42.rpc.hypersync.xyz | | Lukso Testnet | 4201 | https://lukso-testnet.hypersync.xyz or https://4201.hypersync.xyz | https://lukso-testnet.rpc.hypersync.xyz or https://4201.rpc.hypersync.xyz | | Manta | 169 | https://manta.hypersync.xyz or https://169.hypersync.xyz | https://manta.rpc.hypersync.xyz or https://169.rpc.hypersync.xyz | | Mantle | 5000 | https://mantle.hypersync.xyz or https://5000.hypersync.xyz | https://mantle.rpc.hypersync.xyz or https://5000.rpc.hypersync.xyz | | Megaeth | 4326 | https://megaeth.hypersync.xyz or https://4326.hypersync.xyz | https://megaeth.rpc.hypersync.xyz or https://4326.rpc.hypersync.xyz | | Megaeth Testnet2 | 6343 | https://megaeth-testnet2.hypersync.xyz or https://6343.hypersync.xyz | https://megaeth-testnet2.rpc.hypersync.xyz or https://6343.rpc.hypersync.xyz | | Merlin | 4200 | https://merlin.hypersync.xyz or https://4200.hypersync.xyz | https://merlin.rpc.hypersync.xyz or https://4200.rpc.hypersync.xyz | | Metall2 | 1750 | https://metall2.hypersync.xyz or https://1750.hypersync.xyz | https://metall2.rpc.hypersync.xyz or https://1750.rpc.hypersync.xyz | | Mode | 34443 | https://mode.hypersync.xyz or https://34443.hypersync.xyz | https://mode.rpc.hypersync.xyz or https://34443.rpc.hypersync.xyz | | Monad | 143 | https://monad.hypersync.xyz or https://143.hypersync.xyz | https://monad.rpc.hypersync.xyz or https://143.rpc.hypersync.xyz | | Monad Testnet | 10143 | https://monad-testnet.hypersync.xyz or https://10143.hypersync.xyz | https://monad-testnet.rpc.hypersync.xyz or https://10143.rpc.hypersync.xyz | | Moonbeam *(indexing halted, historical data only)* | 1284 | https://moonbeam.hypersync.xyz or https://1284.hypersync.xyz | https://moonbeam.rpc.hypersync.xyz or https://1284.rpc.hypersync.xyz | | Morph | 2818 | https://morph.hypersync.xyz or https://2818.hypersync.xyz | https://morph.rpc.hypersync.xyz or https://2818.rpc.hypersync.xyz | | Opbnb | 204 | https://opbnb.hypersync.xyz or https://204.hypersync.xyz | https://opbnb.rpc.hypersync.xyz or https://204.rpc.hypersync.xyz | | Optimism | 10 | https://optimism.hypersync.xyz or https://10.hypersync.xyz | https://optimism.rpc.hypersync.xyz or https://10.rpc.hypersync.xyz | | Optimism Sepolia | 11155420 | https://optimism-sepolia.hypersync.xyz or https://11155420.hypersync.xyz | https://optimism-sepolia.rpc.hypersync.xyz or https://11155420.rpc.hypersync.xyz | | Plasma | 9745 | https://plasma.hypersync.xyz or https://9745.hypersync.xyz | https://plasma.rpc.hypersync.xyz or https://9745.rpc.hypersync.xyz | | Plume | 98866 | https://plume.hypersync.xyz or https://98866.hypersync.xyz | https://plume.rpc.hypersync.xyz or https://98866.rpc.hypersync.xyz | | Polygon | 137 | https://polygon.hypersync.xyz or https://137.hypersync.xyz | https://polygon.rpc.hypersync.xyz or https://137.rpc.hypersync.xyz | | Polygon Amoy | 80002 | https://polygon-amoy.hypersync.xyz or https://80002.hypersync.xyz | https://polygon-amoy.rpc.hypersync.xyz or https://80002.rpc.hypersync.xyz | | Polygon zkEVM *(indexing halted, historical data only)* | 1101 | https://polygon-zkevm.hypersync.xyz or https://1101.hypersync.xyz | https://polygon-zkevm.rpc.hypersync.xyz or https://1101.rpc.hypersync.xyz | | Robinhood | 4663 | https://robinhood.hypersync.xyz or https://4663.hypersync.xyz | https://robinhood.rpc.hypersync.xyz or https://4663.rpc.hypersync.xyz | | Rootstock | 30 | https://rootstock.hypersync.xyz or https://30.hypersync.xyz | https://rootstock.rpc.hypersync.xyz or https://30.rpc.hypersync.xyz | | Scroll | 534352 | https://scroll.hypersync.xyz or https://534352.hypersync.xyz | https://scroll.rpc.hypersync.xyz or https://534352.rpc.hypersync.xyz | | Sei* | 1329 | https://sei.hypersync.xyz or https://1329.hypersync.xyz | https://sei.rpc.hypersync.xyz or https://1329.rpc.hypersync.xyz | | Sei Testnet* | 1328 | https://sei-testnet.hypersync.xyz or https://1328.hypersync.xyz | https://sei-testnet.rpc.hypersync.xyz or https://1328.rpc.hypersync.xyz | | Sepolia | 11155111 | https://sepolia.hypersync.xyz or https://11155111.hypersync.xyz | https://sepolia.rpc.hypersync.xyz or https://11155111.rpc.hypersync.xyz | | Shimmer Evm | 148 | https://shimmer-evm.hypersync.xyz or https://148.hypersync.xyz | https://shimmer-evm.rpc.hypersync.xyz or https://148.rpc.hypersync.xyz | | Somnia | 5031 | https://somnia.hypersync.xyz or https://5031.hypersync.xyz | https://somnia.rpc.hypersync.xyz or https://5031.rpc.hypersync.xyz | | Somnia Testnet | 50312 | https://somnia-testnet.hypersync.xyz or https://50312.hypersync.xyz | https://somnia-testnet.rpc.hypersync.xyz or https://50312.rpc.hypersync.xyz | | Soneium | 1868 | https://soneium.hypersync.xyz or https://1868.hypersync.xyz | https://soneium.rpc.hypersync.xyz or https://1868.rpc.hypersync.xyz | | Sonic | 146 | https://sonic.hypersync.xyz or https://146.hypersync.xyz | https://sonic.rpc.hypersync.xyz or https://146.rpc.hypersync.xyz | | Sonic Testnet | 14601 | https://sonic-testnet.hypersync.xyz or https://14601.hypersync.xyz | https://sonic-testnet.rpc.hypersync.xyz or https://14601.rpc.hypersync.xyz | | Sophon | 50104 | https://sophon.hypersync.xyz or https://50104.hypersync.xyz | https://sophon.rpc.hypersync.xyz or https://50104.rpc.hypersync.xyz | | Stable | 988 | Access on request - [contact us](https://discord.gg/envio) | Access on request - [contact us](https://discord.gg/envio) | | Superseed | 5330 | https://superseed.hypersync.xyz or https://5330.hypersync.xyz | https://superseed.rpc.hypersync.xyz or https://5330.rpc.hypersync.xyz | | Swell *(indexing halted, historical data only)* | 1923 | https://swell.hypersync.xyz or https://1923.hypersync.xyz | https://swell.rpc.hypersync.xyz or https://1923.rpc.hypersync.xyz | | Tempo | 4217 | https://tempo.hypersync.xyz or https://4217.hypersync.xyz | https://tempo.rpc.hypersync.xyz or https://4217.rpc.hypersync.xyz | | Tron | 728126428 | https://tron.hypersync.xyz or https://728126428.hypersync.xyz | https://tron.rpc.hypersync.xyz or https://728126428.rpc.hypersync.xyz | | Unichain | 130 | https://unichain.hypersync.xyz or https://130.hypersync.xyz | https://unichain.rpc.hypersync.xyz or https://130.rpc.hypersync.xyz | | Worldchain | 480 | https://worldchain.hypersync.xyz or https://480.hypersync.xyz | https://worldchain.rpc.hypersync.xyz or https://480.rpc.hypersync.xyz | | XDC | 50 | Access on request - [contact us](https://discord.gg/envio) | Access on request - [contact us](https://discord.gg/envio) | | XDC Testnet | 51 | Access on request - [contact us](https://discord.gg/envio) | Access on request - [contact us](https://discord.gg/envio) | | Zeta | 7000 | https://zeta.hypersync.xyz or https://7000.hypersync.xyz | https://zeta.rpc.hypersync.xyz or https://7000.rpc.hypersync.xyz | | Zircuit | 48900 | Access on request - [contact us](https://discord.gg/envio) | Access on request - [contact us](https://discord.gg/envio) | | ZKsync | 324 | https://zksync.hypersync.xyz or https://324.hypersync.xyz | https://zksync.rpc.hypersync.xyz or https://324.rpc.hypersync.xyz | | Zora | 7777777 | https://zora.hypersync.xyz or https://7777777.hypersync.xyz | https://zora.rpc.hypersync.xyz or https://7777777.rpc.hypersync.xyz | **Notes:** - **Base Traces***: Traces are available as a paid add-on. Start block: 24000000 (earlier blocks available on request) - **Eth Traces***: Traces are available as a paid add-on - **Injective***: Start block: 129846180 (non-evm before that) - **Sei***: Start block: 79123881 (non-evm before that) - **Sei Testnet***: Start block: 186100000 (non-evm before that) --- --- # Troubleshooting > Fixes for common HyperSync and HyperRPC connectivity issues, including DNS resolution failures. Common connectivity issues with HyperSync and HyperRPC endpoints. If nothing here helps, ask in our [Discord](https://discord.gg/DhfFhzuJQh). ## DNS resolution failures **Symptoms:** requests hang or time out intermittently, the first request after idle is slow but retries work, or you see `Could not resolve host` / `SERVFAIL` for `*.hypersync.xyz` or `*.rpc.hypersync.xyz`. **Cause:** HyperSync endpoints use geographic load balancing with a multi-step DNS delegation chain and short TTLs. Some ISP and home-router resolvers (notably in South Africa and parts of Asia) can't follow the chain and return `SERVFAIL`. This is a client-side resolver issue, not an outage; public resolvers like Cloudflare and Google resolve it correctly. **Verify:** ```bash dig eth.hypersync.xyz A # your system/ISP resolver dig eth.hypersync.xyz A @1.1.1.1 # Cloudflare dig eth.hypersync.xyz A @8.8.8.8 # Google ``` If the first returns `SERVFAIL` and the others return `NOERROR` with IPs, your resolver is the problem. **Fix:** switch your system DNS to public resolvers. - **Linux (systemd-resolved):** ```bash sudo mkdir -p /etc/systemd/resolved.conf.d printf '[Resolve]\nDNS=1.1.1.1 8.8.8.8 1.0.0.1 8.8.4.4\nFallbackDNS=9.9.9.9\n' \ | sudo tee /etc/systemd/resolved.conf.d/dns.conf sudo systemctl restart systemd-resolved ``` Revert by deleting the file and restarting `systemd-resolved`. - **macOS:** `sudo networksetup -setdnsservers Wi-Fi 1.1.1.1 8.8.8.8 1.0.0.1 8.8.4.4` (replace `Wi-Fi` with your interface; revert with `... Wi-Fi Empty`). - **Windows:** Settings > Network & Internet > Wi-Fi > Hardware properties > DNS server assignment > Edit > Manual. Preferred `1.1.1.1`, alternate `8.8.8.8`. - **Docker:** add `dns: ["1.1.1.1", "8.8.8.8"]` to the service in your compose file, or pass `--dns 1.1.1.1 --dns 8.8.8.8` to `docker run`. | Resolver | IPs | | --- | --- | | Cloudflare | `1.1.1.1`, `1.0.0.1` | | Google | `8.8.8.8`, `8.8.4.4` | | Quad9 | `9.9.9.9` | ## Connection timeouts If DNS resolves but requests still hang, check the endpoint directly: ```bash curl -v --max-time 10 https://eth.hypersync.xyz/height ``` If this returns a block height, the service is healthy and the problem is on the network path (typically a corporate/university firewall or transient regional routing). Try from a different network to confirm. ## HTTP 401 Unauthorized or 403 Forbidden HyperSync and HyperRPC need an API token. A request with no token, or a malformed one, gets HTTP 401. A token that is unknown, not yet active, or not allowed to use the product gets HTTP 403. Create a token in the [Envio Dashboard](https://envio.dev/app/api-tokens) and send it with every request. HyperRPC tokens are on their own **HyperRPC Tokens** page in the same sidebar. In a HyperIndex indexer, set it as `ENVIO_API_TOKEN` (see [Environment Variables](https://docs.envio.dev/docs/HyperIndex/environment-variables)). For the exact error messages and fixes, see [API tokens: common errors](https://docs.envio.dev/docs/HyperSync/api-tokens.md#common-errors). ## Getting help Share the output of the `dig` and `curl` commands above, plus your location and ISP, in our [Discord](https://discord.gg/DhfFhzuJQh). --- # Analyzing All Transactions To and From an Address > Explore all transactions, token balances, and approvals for any EVM address with HyperSync. ## Introduction Understanding all transactions to and from an address is an interesting use case. Traditionally extracting this information would be very difficult with an RPC. In this tutorial, we'll introduce you to the [evm-address-summary](https://github.com/enviodev/evm-address-summary) tool, which uses HyperSync to efficiently extract all transactions associated with a specific address. ## About evm-address-summary The [evm-address-summary](https://github.com/enviodev/evm-address-summary) repository contains a collection of scripts designed to get activity related to an address. These scripts leverage HyperSync's efficient data access to make complex address analysis simple and quick. **GitHub Repository**: [https://github.com/enviodev/evm-address-summary](https://github.com/enviodev/evm-address-summary) ## Available Scripts The repository offers several specialized scripts: 1. **All Transfers**: This script scans the entire blockchain (from block 0 to the present) and retrieves all relevant transactions for the given address. It iterates through these transactions and sums up their values to calculate aggregates for each token. 2. **NFT Holders**: This script scans the entire blockchain and retrieves all token transfer events for an ERC721 address. It records all the owners of these tokens and how many tokens they have traded in the past. 3. **ERC20 Transfers and Approvals**: This script scans the blockchain and retrieves all ERC20 transfer and approval events for the given address. It calculates the following: - **Token balances**: Summing up all incoming and outgoing transfers for each token - **Token transaction counts**: Counting the number of incoming and outgoing transactions for each token - **Approvals**: Tracking approvals for each token, including the spender and approved amount ## Quick Start Guide ### Prerequisites - [Node.js](https://nodejs.org/) (v16 or newer) - [pnpm](https://pnpm.io/installation) (recommended) - Git ### Basic Setup 1. **Clone the Repository** ```bash git clone https://github.com/enviodev/evm-address-summary.git cd evm-address-summary ``` 2. **Install Dependencies** ```bash pnpm install ``` 3. **Run a Script** (example with all-transfers) ```bash pnpm run all-transfers 0xYourAddressHere ``` For complete details on all available scripts, their usage, and example outputs, refer to the [project README](https://github.com/enviodev/evm-address-summary#readme). ## Customizing Network Endpoints The scripts work with any network supported by HyperSync. To change networks, edit the `hyperSyncEndpoint` in the appropriate config file: ```typescript // For Ethereum Mainnet export const hyperSyncEndpoint = "https://eth.hypersync.xyz"; ``` For a complete list of supported networks, see our [HyperSync Supported Networks](https://docs.envio.dev/docs/HyperSync/hypersync-supported-networks) documentation. ## Practical Use Cases One powerful application is measuring value at risk for any address, similar to [revoke.cash](https://revoke.cash). You can quickly scan an address to find all approvals and transfers to easily determine any outstanding approvals on any token. This helps identify potential security risks from forgotten token approvals. Other use cases include: - Portfolio tracking and analysis - Auditing transaction history - Research on token holder behavior - Monitoring NFT ownership changes ## Next Steps - Check out the [evm-address-summary repository](https://github.com/enviodev/evm-address-summary) for full documentation - Explore the source code to understand how HyperSync is used for data retrieval - Try modifying the scripts for your specific use cases - Learn more about [HyperSync's capabilities](https://docs.envio.dev/docs/HyperSync/hypersync-query) for blockchain data analysis For any questions or support, join our [Discord community](https://discord.gg/envio) or create an issue on the GitHub repository. --- # Decoding Transaction Traces > Query EVM transaction traces with HyperSync and decode them with viem, to read internal transactions and contract calls that emit no logs. Traces are the calls a transaction makes once it starts running. They are how you see internal ETH transfers, calls that emit no event, and calls that reverted. None of it appears in logs, so an indexer built on events alone never sees it. This tutorial queries traces with HyperSync and decodes the call data with [viem](https://viem.sh), so you can read what a contract was asked to do rather than only what it logged. In a hurry, skip to [the agent prompt](#hand-this-to-your-coding-agent) and let your assistant do it. ## Prerequisites - Node.js 18 or higher - An [API token](https://docs.envio.dev/docs/HyperSync/api-tokens) with trace access Trace data is listed as an on-demand add-on on the [HyperSync pricing page](https://envio.dev/pricing/hypersync), and [supported networks](https://docs.envio.dev/docs/HyperIndex/supported-networks) lists the trace chains as access on request. Ask in [Discord](https://discord.gg/envio) if a trace query comes back without access. ## Traces have their own endpoints Trace data is served from a separate endpoint per chain, so the standard endpoint returns nothing for a trace query. | Chain | Endpoint | | --- | --- | | Ethereum | `https://eth-traces.hypersync.xyz` | | Base | `https://base-traces.hypersync.xyz` | | Gnosis | `https://gnosis-traces.hypersync.xyz` | Point the client at the trace endpoint for the chain you want. These three are the trace chains in [supported networks](https://docs.envio.dev/docs/HyperIndex/supported-networks), so ask in [Discord](https://discord.gg/envio) if you need another. ## Set up ```bash mkdir traces && cd traces npm init -y npm pkg set type=module npm install @envio-dev/hypersync-client viem ``` Put your token in the environment: ```bash export ENVIO_API_TOKEN=your_token_here ``` ## Decode the calls made to a contract This queries every trace whose target is the WETH contract, then decodes the call data against a small ABI. `decodeFunctionData` throws on a selector the ABI does not cover, which is how you separate the calls you care about from the rest. ```javascript import { HypersyncClient, TraceField } from "@envio-dev/hypersync-client"; import { decodeFunctionData, parseAbi } from "viem"; const client = new HypersyncClient({ url: "https://eth-traces.hypersync.xyz", apiToken: process.env.ENVIO_API_TOKEN, }); const WETH = "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"; const abi = parseAbi([ "function transfer(address to, uint256 amount)", "function transferFrom(address from, address to, uint256 amount)", "function withdraw(uint256 amount)", "function deposit()", ]); const fromBlock = 20000000; const res = await client.get({ fromBlock, toBlock: fromBlock + 20, traces: [{ to: [WETH] }], fieldSelection: { trace: [ TraceField.BlockNumber, TraceField.TransactionHash, TraceField.From, TraceField.To, TraceField.Value, TraceField.Input, TraceField.Type, TraceField.Error, ], }, }); const traces = res.data.traces ?? []; console.log(`traces to WETH: ${traces.length}`); for (const t of traces.slice(0, 5)) { try { const { functionName, args } = decodeFunctionData({ abi, data: t.input }); console.log(`${t.transactionHash} ${functionName}(${args.map(String).join(", ")})`); } catch { console.log(`${t.transactionHash} selector ${String(t.input).slice(0, 10)} not in the ABI`); } } ``` `toBlock` is exclusive, so this covers 20 blocks of Ethereum mainnet. They hold 2,893 traces to that one contract, and the decoded lines read like this: ```text traces to WETH: 2893 0xbb4b3fc2b746877dce70862850602f1d19bd890ab4db47e6b7ee1da1fe578a0d transferFrom(0x6b75d8AF000000e20B7a7DDf000Ba900b4009A80, 0x4fc47579eCf6Aa76677ee142B6b75FaF9EeafbA8, 17619098309492736) 0xbb4b3fc2b746877dce70862850602f1d19bd890ab4db47e6b7ee1da1fe578a0d selector 0x70a08231 not in the ABI 0xbb4b3fc2b746877dce70862850602f1d19bd890ab4db47e6b7ee1da1fe578a0d transfer(0x6b75d8AF000000e20B7a7DDf000Ba900b4009A80, 268873804657197056) ``` `0x70a08231` is `balanceOf(address)`, a read the ABI above does not list. Add the signature to `parseAbi` and it decodes too. ## Tracking ETH that moved A trace that carries a `value` moved ETH, including the internal transfers a contract makes, which emit no log. The server filters on `from`, `to`, `address`, `call_type`, `reward_type`, `type` and `sighash`, so value and error are filters you apply to the results yourself. We have a walkthrough of that job on its own, with a `call_type` filter that skips `staticcall` and `delegatecall` before the data leaves the server: [tracking native ETH transfers](https://docs.envio.dev/blog/tracking-native-eth-transfers-hypersync). ## Reading the results - `type` tells you what the trace was. On Ethereum you see `call`, `create` and `suicide`, and a `create` has no `to`. The [query reference](https://docs.envio.dev/docs/HyperSync/hypersync-query) also lists `reward`, and you filter on the same field name, `type` - `error` is set on a call that failed, with values such as `Reverted`, `Out of gas` and `Bad instruction`. Failed calls still appear, which is useful when you are debugging one and noise when you are counting transfers - The Node client gives `value` as a BigInt. The HTTP API returns it as a hex string, so wrap it in `BigInt` if you are calling the endpoint directly - A plain ETH transfer carries `0x` as its input, so there is nothing to decode. A call that sends ETH and calldata together carries both The full list of trace fields and filters is in the [query reference](https://docs.envio.dev/docs/HyperSync/hypersync-query). ## Hand this to your coding agent ```text Using HyperSync, write a Node script that fetches transaction traces from https://eth-traces.hypersync.xyz and decodes them. - Install @envio-dev/hypersync-client and viem, set "type": "module" - Build the client with new HypersyncClient({ url, apiToken: process.env.ENVIO_API_TOKEN }) - Query traces filtered by the contract I name, selecting block number, transaction hash, from, to, value, input, type and error - Decode input with viem's decodeFunctionData against an ABI built by parseAbi, and print the selector for any call the ABI does not cover - Skip traces that carry an error, and treat value as a BigInt Docs: https://docs.envio.dev/docs/HyperSync/tutorial-decoding-traces ``` ## Next steps - [Query reference](https://docs.envio.dev/docs/HyperSync/hypersync-query) for every trace field and filter - [Tracking native ETH transfers](https://docs.envio.dev/blog/tracking-native-eth-transfers-hypersync) for streaming value-carrying traces - [Analyzing address transactions](https://docs.envio.dev/docs/HyperSync/tutorial-address-transactions) for the same job across logs and transactions - [Stream config and tuning](https://docs.envio.dev/docs/HyperSync/stream-config-tuning) for streaming larger ranges rather than a single query ---