Skip to main content

Key-Value Store in Streams Filters

Updated on
Jul 20, 2026

Quicknode's Key-Value Store product can be accessed directly from your Streams filters using the qnLib methods (JavaScript) or the qn SDK (Go). This enables powerful use cases like maintaining watchlists, storing contract ABIs, and building stateful filters that remember data across blocks.

Why Use Key-Value Store in Filters?​

Without Key-Value Store, filters are stateless—they process each block independently with no memory of previous blocks. Key-Value Store changes this by giving your filters access to persistent storage:


  • Watchlists — Maintain lists of addresses to monitor and check transactions against them
  • Contract ABIs — Store ABIs externally so you can update them without modifying filter code
  • State tracking — Remember values across blocks (e.g., track cumulative totals, flag seen transactions)
  • Dynamic configuration — Change filter behavior by updating stored values, no redeployment needed

Note

Go filters have full access to the Key-Value Store via the qn SDK — reads: qn.ContainsListItem, qn.ContainsListItems, qn.GetValue, qn.GetList, qn.GetAllKeys, qn.GetAllLists; writes: qn.AddListItem, qn.RemoveListItem, qn.DeleteList, qn.AddValue, qn.DeleteValue. Go calls are synchronous (no await needed), and write methods return a Go error directly — always check it.

All qnLib methods are asynchronous and return Promises. Always use the await keyword when calling these methods, and ensure your filter's main function is declared with async.

Data Types: Lists vs Values​

Key-Value Store offers two data structures, each suited to different use cases:

TypeWhat it storesBest for
ListsCollections of items (e.g., addresses)Watchlists, allowlists, blocklists
ValuesKey-value pairsABIs, configuration, counters

Working with Lists​

Lists are ideal for maintaining collections of items you want to check against streaming data—like a watchlist of wallet addresses.

Available Methods​


Go filters access lists through the qn SDK. All calls are synchronous, and the write methods return a Go error you should check:

MethodDescription
qn.AddListItem(listName, item)Add single item to a list. Returns error.
qn.RemoveListItem(listName, item)Remove single item from a list. Returns error.
qn.ContainsListItem(listName, item)Check a single item for membership. Returns bool.
qn.ContainsListItems(listName, items)Batch check many items at once. Returns []bool index-aligned with items. Always batch lookups to reduce round trips.
qn.GetList(listName)Returns every item in a list. Returns ([]string, error).
qn.GetAllLists()Returns every list name for the account. Returns ([]string, error).
qn.DeleteList(listName)Deletes a list. Returns error.

Example: List Operations​

This example demonstrates all list operations:


type Payload struct {
	Data     []interface{}          `json:"data"`
	Metadata map[string]interface{} `json:"metadata"`
}

func Filter(qn *qn.QNContext, payload Payload) interface{} {
	const listName = "list_docs_example"

	// Results for each operation
	results := map[string]interface{}{}

	// Add items to the list
	if err := qn.AddListItem(listName, "item1"); err != nil {
		results["addItem1Error"] = err.Error()
	}
	if err := qn.AddListItem(listName, "item2"); err != nil {
		results["addItem2Error"] = err.Error()
	}

	// Remove an item from the list
	if err := qn.RemoveListItem(listName, "item1"); err != nil {
		results["removeItemError"] = err.Error()
	}

	// Read every item in the list
	if items, err := qn.GetList(listName); err != nil {
		results["getListError"] = err.Error()
	} else {
		results["items"] = items
	}

	// Batch check multiple items for list membership (one round trip)
	results["contains"] = qn.ContainsListItems(listName, []string{"item1", "item2"})

	// List every list name for the account
	if lists, err := qn.GetAllLists(); err != nil {
		results["getAllListsError"] = err.Error()
	} else {
		results["lists"] = lists
	}

	// Delete the list
	if err := qn.DeleteList(listName); err != nil {
		results["deleteListError"] = err.Error()
	}

	return results
}

Working with Values​

Values store key-value pairs—perfect for configuration values, contract ABIs, or any data you want to retrieve by key.

Available Methods​


In Go, values are accessed through the qn SDK. All calls are synchronous, and write methods return a Go error you should check:

MethodDescription
qn.AddValue(key, value)Creates or updates a key-value pair. Returns error.
qn.GetValue(key)Retrieves the value for a key. Returns (string, error).
qn.GetAllKeys()Lists every key for the account. Returns ([]string, error).
qn.DeleteValue(key)Deletes a key-value pair. Returns error.

Example: Value Operations​

This example demonstrates all value operations:


type Payload struct {
	Data     []interface{}          `json:"data"`
	Metadata map[string]interface{} `json:"metadata"`
}

func Filter(qn *qn.QNContext, payload Payload) interface{} {
	// Results for each operation
	results := map[string]interface{}{}

	// Create (or update) a key-value pair
	if err := qn.AddValue("set_docs_example", "value1"); err != nil {
		results["addValueError"] = err.Error()
	}

	// Read a value by key
	if value, err := qn.GetValue("set_docs_example"); err != nil {
		results["getValueError"] = err.Error()
	} else {
		results["value"] = value
	}

	// Bulk add — no batch method in Go, so loop
	for key, value := range map[string]string{
		"set_docs_example2": "value1",
		"set_docs_example3": "value2",
	} {
		if err := qn.AddValue(key, value); err != nil {
			results["addValueError_"+key] = err.Error()
		}
	}

	// List all keys for the account
	if keys, err := qn.GetAllKeys(); err != nil {
		results["getAllKeysError"] = err.Error()
	} else {
		results["keys"] = keys
	}

	// Delete key-value pairs
	if err := qn.DeleteValue("set_docs_example2"); err != nil {
		results["deleteValueError2"] = err.Error()
	}
	if err := qn.DeleteValue("set_docs_example3"); err != nil {
		results["deleteValueError3"] = err.Error()
	}

	return results
}

Managing ABIs with Key-Value Store​

You can store and manage your contract ABIs using Key-Value Store:


  1. Upload ABIs using REST APIs
  2. Update ABIs when contracts are upgraded
  3. Load multiple ABIs dynamically in your filter
  4. Share ABIs across different streams

Common Pitfalls​

The pitfalls below are specific to JavaScript's Promise-based qnLib API. Go's qn SDK is synchronous — there are no Promises to await, so these issues don't apply to Go filters. In Go, the equivalent discipline is to check the error returned by every write method (qn.AddListItem, qn.RemoveListItem, qn.DeleteList, qn.AddValue, qn.DeleteValue) and to batch membership checks with qn.ContainsListItems rather than calling qn.ContainsListItem in a loop.

Missing await Keyword​

All qnLib methods are asynchronous and return Promises. Failing to use await will result in unexpected behavior:

// INCORRECT - returns a Promise, not the actual result
function main() {
  const result = qnLib.qnContainsListItems('myList', ['item']);
  // result is a Promise object, not the boolean array
  return result;
}

// CORRECT - returns the resolved value from the Promise
async function main() {
  const result = await qnLib.qnContainsListItems('myList', ['item']);
  // result contains the index-aligned boolean array
  return result;
}

Boolean Checks with Async Methods​

When checking list membership, use qnContainsListItems and batch your lookups. Be careful to await the result when using it in conditionals:

// INCORRECT - checks if Promise exists, not the result
function main() {
  if (qnLib.qnContainsListItems('myList', ['item'])) {
    // This will always evaluate to true regardless of whether
    // the item is in the list or not!
    return { exists: true };
  }
  return { exists: false };
}

// CORRECT - batch lookup, then use the index-aligned boolean array
async function main() {
  const hits = await qnLib.qnContainsListItems('myList', ['item']);
  if (hits[0]) {
    return { exists: true };
  }
  return { exists: false };
}

Error Handling for Large Operations​

When working with large datasets, implement proper error handling:

// INCORRECT - no error handling for large list operations
async function main() {
  // This might fail with large lists
  await qnLib.qnUpsertList('myList', { add_items: veryLargeArray });
  return { success: true };
}

// CORRECT - handles potential size-related errors
async function main() {
  try {
    // Use chunking for large operations
    const chunkSize = 50000;
    for (let i = 0; i < veryLargeArray.length; i += chunkSize) {
      const chunk = veryLargeArray.slice(i, i + chunkSize);
      await qnLib.qnUpsertList('myList', { add_items: chunk });
    }
    return { success: true };
  } catch (error) {
    return { error: `Failed to update list: ${error.message}` };
  }
}

Cascading Async Operations​

Chain operations carefully to avoid cascading failures:

// INCORRECT - continues execution even if a prerequisite operation fails
async function main() {
  await qnLib.qnAddListItem('myList', 'important_item');
  const hits = await qnLib.qnContainsListItems('myList', ['important_item']);
  return { found: hits[0] };
}

// CORRECT - checks intermediate results before continuing
async function main() {
  try {
    const addResult = await qnLib.qnAddListItem('myList', 'important_item');
    if (!addResult) {
      return { error: "Failed to add item to list" };
    }
    const hits = await qnLib.qnContainsListItems('myList', ['important_item']);
    return { found: hits[0] };
  } catch (error) {
    return { error: `Operation failed: ${error.message}` };
  }
}

Performance: Batch List Lookups​

Batch all list membership checks into as few calls as possible to reduce round trips to the Key-Value Store. This batch lookup is optimized for performance.

  • JavaScript: qnLib.qnContainsListItems(listId, items[]) → Promise<boolean[]> (index-aligned to items)
  • Go: qn.ContainsListItems(listId, items []string) []bool (index-aligned to items, synchronous)

// Gather all addresses, then one batched lookup — minimizes round trips.
type Payload struct {
	Data     []Block                `json:"data"`
	Metadata map[string]interface{} `json:"metadata"`
}
type Block struct {
	Transactions []Transaction `json:"transactions"`
}
type Transaction struct {
	From string `json:"from"`
	To   string `json:"to"`
}

func Filter(qn *qn.QNContext, payload Payload) interface{} {
	const listID = "WATCHLIST_EVM_MAINNET"

	// Gather from/to, de-duped, before the single batch call
	seen := make(map[string]bool)
	var addrs []string
	for _, block := range payload.Data {
		for _, tx := range block.Transactions {
			if tx.From != "" {
				from := strings.ToLower(tx.From)
				if !seen[from] {
					seen[from] = true
					addrs = append(addrs, from)
				}
			}
			if tx.To != "" {
				to := strings.ToLower(tx.To)
				if !seen[to] {
					seen[to] = true
					addrs = append(addrs, to)
				}
			}
		}
	}
	if len(addrs) == 0 {
		return nil
	}

	hits := qn.ContainsListItems(listID, addrs)

	// Fast lookup map (index-aligned to addrs)
	hitMap := make(map[string]bool, len(addrs))
	for i, addr := range addrs {
		hitMap[addr] = hits[i]
	}

	// Return matched txs
	var matched []Transaction
	for _, block := range payload.Data {
		for _, tx := range block.Transactions {
			if hitMap[strings.ToLower(tx.From)] || hitMap[strings.ToLower(tx.To)] {
				matched = append(matched, tx)
			}
		}
	}

	if len(matched) == 0 {
		return nil
	}
	return map[string]interface{}{"data": matched}
}

When you need to check list membership for multiple values (e.g. many tx.from/tx.to or many log addresses), collect them, de-duplicate, then call qnContainsListItems (JavaScript) or ContainsListItems (Go) once and use the returned boolean array. Avoid multiple per-item lookups.

Last updated on
Share this doc