Quicknode

March 31, 2024 — 4 min read

TL:DR

• Enjoy faster processing and more flexible filtering capabilities

• Streams filters now support asynchronous operations with Key-Value Store

• New filters billing introduces a 2.5KB minimum for filtered blocks

• Migrate existing streams by following our updated docs before the deadline

Quicknode Streams revolutionized how developers interact with blockchain data, allowing you to filter, transform, and route only the specific data points that matter to your application.

Today, we're excited to announce significant enhancements to Streams filters that will give your data processing pipeline even more power and flexibility.

What's Changing: More Powerful Filters

Asynchronous Operations

Filters now support asynchronous logic, enabling you to make real-time decisions based on data from our Key-Value Store. This unlocks entirely new workflows for your applications:

// Before: Synchronous operations only
if (qnContainsListItem("my-list", tx.hash)) {
  return tx;
}

// Now: Asynchronous operations with await, qnLib reference
if (await qnLib.qnContainsListItem("my-list", tx.hash)) {
  return tx;
}

Faster Data Lookups

The new filter engine delivers performance improvements when interacting with Key-Value Store:

• qnLib.qnGetList: Retrieve stored values with up to 2x lower latency

• qnLib.qnUpsertList: Update your lists with up to 2.5x faster operation times

• qnLib.qnContainsListItem: Check for data presence with up to 60% reduced processing time

Enhanced Monitoring

Track filter performance with our new metrics dashboard section, giving you insights into:

• Filter execution durations (p99, p90, p50 metrics)

• Processing latency statistics

• Detailed views of processed and filtered blocks

Billing Updates to Support Enhanced Infrastructure

We're also updating our billing model for filtered blocks to maintain the high-performance infrastructure needed for these improvements.

Minimum Data Charge

Starting April 21, 2025, filtered blocks will have a minimum billable size of 2.5KB, even if the filtered payload is smaller. This change:

• Only affects blocks processed through filters

• Doesn't apply to non-filtered data or empty Solana slots

• Ensures sustainable infrastructure for all customers

The minimum charge reflects the computational resources required to process blocks through your custom filter logic, regardless of the amount of data ultimately delivered.

Network-Specific Considerations

Different blockchain networks produce blocks at varying frequencies, which impacts your minimum monthly usage when filtering blocks.

High-Frequency Networks (>20GB/month):

• Solana: 24.1GB/month

• Arbitrum: 24.1GB/month

• Arbitrum Nova: 24.1GB/month

Medium-Frequency Networks (2-5GB/month):

• Avalanche C-Chain: 3.02GB/month

• Base: 3.02GB/month

• Imx: 3.02GB/month

• Optimism: 3.02GB/month

• Polygon: 3.02GB/month

• BNB Chain: 2.01GB/month

• Scroll: 2.01GB/month

Lower-Frequency Networks (<2GB/month):

• Polygon zkEVM: 1.51GB/month

• Fantom: 1.51GB/month

• Bera: 1.21GB/month

• Celo: 1.21GB/month

• Gnosis: 1.21GB/month

• Ethereum: 0.50GB/month

These figures represent minimum usage while streaming at the tip of the chain. Actual data charges may be higher depending on your filter logic, payload sizes, and historical data backfilling.

Check your current usage patterns and consider upgrading your plan to take advantage of higher tiers with better compression capabilities and lower overage rates.

Migration Timeline

• April 1, 2025: All new streams will use the enhanced filter engine and billing model

• April 21, 2025: Existing streams will transition to the new billing model

The deadline to migrate existing Streams to the new filter engine will be announced soon.

How to Migrate Your Existing Streams

The in-app migration flow will be available soon to streamline this process. Here's how to migrate your streams today:

1. Pause your existing Stream in the Stream settings

2. Create a new Stream with the same configuration, starting from a block where you paused your existing Stream.

3. Update your filter code. Check our updated Key-Value Store docs for Filters to see how to migrate your filter code.

4. Test and deploy your new Stream.

For detailed step-by-step instructions, refer to our comprehensive guide.

The new Filters section in your Streams metrics dashboard provides unprecedented visibility into your filter performance:

• Count of sub-threshold blocks (under 2.5KB)

• Filter latency statistics

These insights help you optimize filter strategies for both performance and cost efficiency.

Get Started with Streaming

Ready to experience the future of Streams filters? Create a new stream today to start using the enhanced filters immediately. For existing streams, plan your migration to take advantage of these improvements before the deadline.