Marketplace has launched, further enabling blockchain developers! Learn more

How to Fetch Transaction History on Ethereum using Web3.py

September 09, 2022

Overview

If you are building on Ethereum, you may run into scenarios where you need to fetch transaction history for a given address. Unfortunately, the current design for Ethereum lacks an easy way to retrieve this data. For this reason, we are dedicating this guide to explaining the different ways in which you can retrieve transaction history on Ethereum. We will demonstrate one of these ways with Python. Take a look at the prerequisites for what you'll need to follow along: 

What You Will Need

Methods to Retrieve Transactions for a Given Address

To retrieve transactions for a given address, we can do any of the following: 

  • Iterate through each block: Since there is no specific RPC method we can call to obtain a list of transaction data for a given address, we will need to iterate through the blocks of the blockchain ourselves and record each instance where we detect a transaction related to our address. This method can be time-consuming and can consume a lot of RPC calls if you need to start from the beginning of the blockchain (genesis block). Additionally, we should request this data from an Archive node (which contains all the historical data of the blockchain since the genesis block).

  • Use an indexing service: APIs such as the Etherscan API allow you to fetch transactions for a given address. However, these come with limitations as you will need to make different requests for different types of transactions you are looking for (e.g., external vs. internal transactions).

  • Use an open-sourced Indexer: This repository has set up an indexing database for users to be able to query this type of transaction data. Although it takes a bit longer to set up, it will be easier to analyze and manage once you have the data.

Access an Ethereum Node with QuickNode

If we're interested in getting all blocks from genesis, we'll need to communicate with an Archive node. At the current time of this writing, the full state of the Ethereum blockchain takes around 10,000 GB of storage (~11TB). Luckily, we don't need to manage this infrastructure ourselves, and we can use QuickNode to boot up an Archive node.

To access an Ethereum node, navigate to the endpoints page and click the Create an Endpoint button. Then, select the chain of your choice. For this example, we are using the Ethereum Mainnet, but you can also use other EVM-related chains (i.e., Fantom, Polygon, Avalanche, Optimism).

QuickNode Select Blockchain Tab


If you're looking to get transaction data from the very beginning of the Ethereum chain, select the Archive node add-on:

QuickNode Configure Add-ons Tab


Note: If you're looking to include transactions where your address was included in a trace call, QuickNode provides a quick and easy way to enable this functionality on your blockchain node via the Trace Mode add-on.

Once your node is ready, copy the HTTP URL as you will need it in the following section.

Fetching Transaction History using Web3.py

Now that we have our Archive node ready, we will move on to creating the Python script. Open your terminal and run the following commands to create a project directory:

mkdir eth-transactions-web3py && cd eth-transactions-web3py && echo > transaction_finder.py

Next, we will install the required dependencies:

pip install web3

For simplicity, this demonstration will only be searching from the current block and the trailing 100 blocks. Additionally, we'll only check if the to address matches the one set in our blockchain_address variable. In your transaction_finder.py file, add the following code:

fetching transaction history using web3 py

Copy 
#import dependencies
import pickle
from web3 import Web3, HTTPProvider

#instantiate a web3 remote provider
w3 = Web3(HTTPProvider('YOUR_QUICKNODE_HTTP_ENDPOINT'))

#request the latest block number
ending_blocknumber = w3.eth.blockNumber

#latest block number minus 100 blocks
starting_blocknumber = ending_blocknumber - 100 

#filter through blocks and look for transactions involving this address
blockchain_address = "INPUT_A_VALID_ETHEREUM_ADDRESS"

#create an empty dictionary we will add transaction data to
tx_dictionary = {}

def getTransactions(start, end, address):
    '''This function takes three inputs, a starting block number, ending block number
    and an Ethereum address. The function loops over the transactions in each block and
    checks if the address in the to field matches the one we set in the blockchain_address.
    Additionally, it will write the found transactions to a pickle file for quickly serializing and de-serializing
    a Python object.'''
    print(f"Started filtering through block number {start} to {end} for transactions involving the address - {address}...")
    for x in range(start, end):
        block = w3.eth.getBlock(x, True)
        for transaction in block.transactions:
            if transaction['to'] == address or transaction['from'] == address:
                with open("transactions.pkl", "wb") as f:
                    hashStr = transaction['hash'].hex()
                    tx_dictionary[hashStr] = transaction
                    pickle.dump(tx_dictionary, f)
                f.close()
    print(f"Finished searching blocks {start} through {end} and found {len(tx_dictionary)} transactions")
    

getTransactions(starting_blocknumber, ending_blocknumber, blockchain_address)

Note that the code above stores the transaction data in a pickle file. The advantage of storing the data in pickle format is that it can serialize any Python object without handling additional code. Plus, we can then easily read, analyze and clean up the results without making another request to the blockchain for the same information.

Remember to add your RPC URL to the w3 variable and also update the blockchain_address to the address you're searching historical transactions for. When you're finished with that, save the file and run the script by running the command python3 transaction_finder.py

Terminal output from transactionfinder.py script


If the output in your terminal is similar to the screenshot above, congrats! 🎉

To read the transactions.pkl file's contents, you can create another file called transaction_reader.py and add the following code snippet:

fetching transaction history using web3 py

Copy 
import pickle

data = pickle.load( open("transactions.pkl", "rb" ))
print(data)

When running the transaction_reader.py script, you should get something similar to the screenshot below, where the key is the transaction hash, and the value contains the transaction details.

Terminal output from transaction_reader.py script


Important things to note: The script above only detects transactions with your address in the to or from field. There are scenarios where a transaction can trigger smaller actions that will still modify the internal state of the EVM. The data related to these actions are logged internally within the transaction and are classified as a trace.

If you're looking for transactions related to NFTs, you can use our NFT API to pull transaction history and key collection information about NFTs without having to sort through individual smart contracts.

Conclusion

There you have it! We just demonstrated how to fetch transactions for a given address on Ethereum! If you find this useful, check out some of our guides here. Subscribe to our newsletter for more articles and guides. Feel free to reach out to us via Twitter if you have any feedback. You can always chat with us on our Discord community server, featuring some of the coolest developers you'll ever meet 😊

Related articles 35

How to do a non-custodial transaction with QuickNode
Originally Published On: Jun 19, 2021
Updated On: Sep 9, 2022

Private keys are one of the most... A non-custodial wallet is a... Custodian of Private... At We could use pretty much any... We could use any of the many... Let us create a token transfer... Congratulations on making a...

Continue reading
How to Set Up a Near Project from Scratch
Originally Published On: Jan 27, 2022
Updated On: Sep 9, 2022

In this tutorial we will look at... In this section, we will... In this section, we will look at... In this section, we will look at... When writing smart contracts on... In the last section, we looked... In the last section, we looked... In the last section, we looked... We have finally come to the end...

Continue reading
Estimating gas price using pending transactions in Python
Originally Published On: Feb 20, 2021
Updated On: Sep 9, 2022

To send a transaction on the... Gas is the unit to measure the... Why do we want to estimate gas?... Our first step here would be to... For our purposes today, we could... The first thing we’re going to... Now you know how to estimate gas...

Continue reading
Como crear y lanzar un ERC-721 (NFT)
Originally Published On: Dec 27, 2021
Updated On: Sep 9, 2022

Coleccionables digitales que son... Fungible significa que es lo... ERC es la abreviación para... Ahora que conocemos qué es y... Antes de escribir nuestro... Felicidades por crear tu propio...

Continue reading
How to Access Bitcoin Mempool
Originally Published On: Aug 16, 2021
Updated On: Sep 21, 2022

Bitcoin is the father of... The transactions which are sent... For our purpose today, we need a... We will use Bitcoin RPC methods... Congratulations on mastering the...

Continue reading
How to Mint an NFT on Polygon with Ethers.js
Originally Published On: Mar 28, 2022
Updated On: Sep 9, 2022

This guide demonstrates how to... Ethers.js is a complete Ethereum... As a reminder, the act of... We will deploy our contract on... Time to write the code! Start by... Note, you will need some... That’s it! You have minted an...

Continue reading
How to access Ethereum Mempool
Originally Published On: Oct 15, 2020
Updated On: Sep 15, 2022

On Ethereum, when a transaction... In blockchain terminology, a... For the purpose of this guide,... First, query the Let’s examine how to subscribe... This short guide showed you how...

Continue reading
The Web3 Developer Stack
Originally Published On: Jul 3, 2021
Updated On: Sep 15, 2022

A developer stack is a bag of... If we can imagine web3 as a car... The Chainlink price feed... Head over to the We’ll need a Kovan node to... We’ll now write a short... Congratulations :D You’ve taken...

Continue reading
How to Mint NFTs on the Stacks Blockchain
Originally Published On: Aug 19, 2022
Updated On: Sep 9, 2022

Stacks is an emerging blockchain... The Stacks ecosystem has adopted... Open up a terminal window and... Implementing the SIP-009... This section will demonstrate... The moment you've been waiting... Awesome Work! You’ve learned how...

Continue reading
Introduction to Scaffold-ETH 🏗
Originally Published On: Oct 7, 2021
Updated On: Sep 9, 2022

Developing applications involves... First of all, before getting... Scaffold-ETH comes with a demo... We will set up a QuickNode... Congratulations, and thank you...

Continue reading
How to connect to Ethereum using PHP
Originally Published On: Jun 11, 2021
Updated On: Sep 9, 2022

PHP is a very popular choice... Before installing the web3.php,... We could use pretty much any... Now, make a PHP script file... Congratulations on connecting to...

Continue reading
How to use Subspace with QuickNode
Originally Published On: Sep 8, 2020
Updated On: Sep 9, 2022

In this guide, we'll understand... To install Subspace, we'll need... For our purposes today, we could... Now let's create a short script,... Run the file using the node...

Continue reading
How to Connect Your Dapp With MetaMask Using Ethers.js
Originally Published On: Dec 13, 2021
Updated On: Sep 9, 2022

In our dApp, we will have a... A dApp is an application that... We will need to install the... Ethers.js is a lightweight... With this application, we will... Next, we will create a function... After connecting to MetaMask, we... To handle the wallet address of... This section will show us how to... In this tutorial, we...

Continue reading
How to Lazy Mint an NFT on Rarible with Rarepress
Originally Published On: Feb 23, 2022
Updated On: Sep 9, 2022

NFTs are great for creators to... To mint an NFT, you write data... Lazy Minting is a process in... We will write some JavaScript... Rarepress interacts with the... Now that we have everything in... Congratulations on listing your...

Continue reading
How to get the balance of an ERC-20 token
Originally Published On: Jun 14, 2021
Updated On: Sep 21, 2022

When a new token is made on the... You could use any Ethereum... Now that you've done the legwork... Getting the ERC20-Token Balance... In the top of your... ABI is short for You will most likely interact... You can use a similar process... We now have a connection to an... Having made it to the end, you...

Continue reading