Abraar Ahmed

March 14, 2023 — 11 min read

The Web3 domain is rapidly progressing every day. However, developers and even end-users feel a void because the underlying blockchain infrastructure is not quick enough to provide an ideal user experience (UX) like that of Web2.

In the previous article, "Justifying Quick in Quicknode: A Response Time Comparison of Blockchain Node Providers," we discussed the importance of application performance for Web3 adoption. Additionally, we compared the response times of various blockchain node providers using an ideal model to measure the fastest provider for Ethereum and other EVM-based blockchains (Polygon and BNB Smart Chain). Finally, we concluded that Quicknode is the quickest and most stable blockchain node provider.

That said, not all blockchains are equal. Every chain has a different architecture and, hence, unique requirements. The performance results of Chain A do not hold true for Chain B, and vice versa. While Ethereum and EVM-based blockchains are somewhat similar, a blockchain like Solana is a different ballgame altogether.

In this article, we will compare the performance and data reliability of blockchain node providers to determine who provides the best infrastructure experience for the Solana blockchain.

The Rise of Solana

Countless blockchains have come and gone, but none have stood the test of time like Ethereum. That's why Ethereum is considered the "gold standard" of programmable blockchains. While every other blockchain yearned to be the "Ethereum killer," Solana, the new kid on the block, took a different approach.

In 2017, Solana's founder, Anatoly Yakovenko, outlined a novel approach for blockchain timekeeping, called Proof of History (PoH), in his draft (later published as the Official Whitepaper). Eventually, PoH and other vital innovations enabled Solana to achieve a throughput of 65,000 transactions per second (TPS), with a theoretical throughput of 710,000 TPS in the future.

Solana's innovative approach has opened many doors for blockchain scalability, but has also introduced new challenges, such as managing blockchain infrastructure.

Node Providers as Deliverers

Running a Solana node to communicate with the blockchain is a demanding task. It requires powerful hardware (a minimum of 16 cores CPU with 256 GB RAM) and constant maintenance to keep up with the block time of 400 milliseconds. A small slip can lead to an unreliable local state of the blockchain.

This is where Solana node providers come to the rescue. They run and manage Solana nodes in-house and provide access to developers through an API endpoint.

Not All Providers Are Equal

As mentioned earlier, Solana is different from other blockchains. Running a node with the validator software and exposing it over an API is not sufficient, as Solana's architecture is sophisticated.

At a minimum, it requires system optimization, constant monitoring, and management of long-term persistent storage. The first two are straightforward, but managing storage with a BigTable instance is not.

While the Solana Foundation team provides a publicly accessible BigTable, it's not ideal for running self-hosted nodes. So, when running your own BigTable instance, factors such as hardware quality play a crucial role in the overall RPC performance.

As we dig deeper, differences between Solana node providers become more visible due to such factors. Therefore, it's important to compare the performance of these node providers with a realistic model and an unbiased approach to choose the best provider for your needs. In the following sections, we will do just that.

Comparing Solana Response Times

We discussed in the previous article that response time is not solely latency, but the sum of the round-trip time (RTT) and the service time. That is, the total time taken for a request to reach the destination, be processed, and sent back to the source as a response.

To measure accurate response time with complete fairness, we will use the same comparison model used earlier with these seven essential points:

1. Using paid accounts.

2. Calling RPC methods that are not easily cacheable.

3. Using multiple cloud providers.

4. Testing from multiple locations.

5. Making concurrent test calls.

6. Conducting continuous and frequent tests.

7. Keeping the data untampered.

Running The Response Time Tests

We have an internal comparison framework called QuickLee ("Quickly"), which is built using the above model and is being used to run these tests. It ticks all the boxes:

1. We use API endpoints created through paid accounts for the following providers:
   a. Alchemy
   b. Ankr
   c. Chainstack
   d. Pokt Network (Registered, not paid, as no self-serve option available)
   e. Quicknode

2. We make test calls using the getBalance RPC method to query the balance of a specific wallet address. The wallet address used as a parameter remains the same for all tests.

3. We use multiple cloud providers where Quicknode doesn't host its own infrastructure (while the others do, which is acceptable to us):
   a. Amazon Web Services (AWS)
   b. Digital Ocean (DO)

4. We set up multiple servers in various geographical locations globally:
   a. AWS
   i. Ohio (US-E)
   ii. Northern California (US-W)
   iii. Frankfurt (EU)
   iv. Tokyo (AP)
   b. DO
   i. New York (US-E)
   ii. London (EU)
   iii. Singapore (AP)

5. We created a script using Go that makes concurrent test calls to all providers.

6. We make test calls every 10 seconds, which means we have 6 data points for each provider every minute.

7. We pipe the test data without any interference into DataDog. Each test, or call, records the total response time along with other metadata, such as the response status code.

Finally, we create multiple charts in Datadog to answer various questions. The charts reflect the results in the selected timeframe.

We have created an Overview table with multiple columns:

• (Provider) Name

• Average Response Time aggregated by Datadog using the response time data points.

• Total Calls

• Failed Calls

• Availability calculated using Total Calls und Failed Calls: ((Total Calls - Failed Calls) / Total Calls) * 100

Every region-specific section has:

• Timeseries Graph drawn using the average response time. We have applied a 5-point rolling median function to the chart to remove outliers and cleanly visualize the stability.

• Top List populated based on the average response time in the region.

Which Solana Node Provider Is The Fastest?

For the comparison in the article, we will use data from Jan 22, 2023, 00:00 UTC to Feb 18, 2023, 00:00 UTC (4 weeks). We have more than 1 million data points (for each provider) in this timeframe, which is sufficient to make an unbiased and clear comparison.

In terms of availability, all providers were able to hit the 99.9% mark, except for Ankr (99.6%). Chainstack and Quicknode had the least failed calls, with 14 and 51 calls, respectively.

Quicknode had the fastest global average response time at 104 ms, followed by Chainstack and Alchemy with 361 ms and 401 ms, respectively. Overall, Quicknode was almost 3.5x faster than the next best provider.

Average Response Time (Global)

Moreover, Quicknode appeared to be the most stable with minimal fluctuations globally.

Quicknode was also the quickest provider across all regions, with an average response time of 115 ms, 54 ms, and 139 ms in the US, EU, and AP, respectively.

In the US, Quicknode and Chainstack were the only providers with a response time under 200ms. In the EU, the difference between Quicknode and the Chainstack (runner-up) was almost 6x, with 54 ms and 300 ms, respectively. In AP, all providers except Quicknode struggled. While Quicknode had an average response time of 139 ms, the next provider in line, Chainstack, had an average response time of 699 ms.

The results above demonstrate that Quicknode is the top-performing provider globally and across all regions. Chainstack offers decent performance, followed by Alchemy, but they are still far behind.

Verifying Data Reliability

One of the biggest challenges faced by blockchain node providers and developers is data reliability. Even if you receive super-fast response times, stale data can degrade your application's performance. This is especially true for applications that use an extremely fast blockchain like Solana.

For example, Solana checks for the latest blockhash before sending a transaction, and if the data is even a minute old, the transaction won't go through. Thus, data reliability is of utmost significance.

Imagine operating a crypto wallet or running an NFT mint website. An instance of unreliable data can break your product's functionality. And if this happens too often, your customers will start looking for alternatives.

The Data Reliability Verification Model

Any data reliability verification model needs to check three essential things:

1. Is the data up-to-date?

2. Is there any lag in receiving the latest data?

3. Does the data update regularly and frequently?

We created an open-source tool, Solana Block Derby, that uses these three key pieces, along with the parts of the framework used for response time comparison. It queries the latest slot number from Solana node providers concurrently, calculates the winner(s), and stores them in a PostgreSQL database. At any point, you can query the database to find the answer to the above questions.

Running the Data Reliability Test

We ran Solana Block Derby from Feb 26, 2023, to March 1, 2023, with a timeout of 3 seconds and an interval of 5 seconds. The tool was run in parallel from three locations covering the US, EU, and AP. The response data was inserted into a database in real-time, and we collected 100,000 entries for each provider by the end of the test.

Which Solana Node Provider Is The Most Reliable?

To accurately measure data reliability, we need to account for all three factors listed above. So, let’s examine each of them one by one.

For the latest available slot data, Quicknode won the most rounds with 77,845 wins, followed by Chainstack with 56,978 wins, and Pokt Network with 54,508 wins. This means that out of every 100 calls, Quicknode provides the latest slot data 78 times.

Timeout results were similar, with Alchemy timing out the least with 67 calls, followed by Quicknode with 70 calls, and Chainstack with 108 calls. Pokt Network timed out 16,326 calls and Ankr timed out 25,531 calls, meaning they timed out 20-30 times out of every 100 calls, demonstrating substantial lag in sending the latest data.

As for updated data frequency, Pokt Network had the lowest duplicate percentage with 12.95%, followed by Quicknode with 15.58%, and Chainstack with 16.29%. Ankr and Alchemy had the highest duplicate percentages with 23.01% and 23.11%, respectively, indicating that they returned the same value 23 times out of every 100 calls, which suggests stale data.

Combining all three factors, Quicknode had the most wins with negligible timeouts and fewer duplicates compared to the others, exhibiting superior data reliability. Chainstack loosely followed Quicknode in terms of data reliability with fewer wins, similar timeouts, and duplicate percentages.

Quicknode Provides The Best Solana Infrastructure Experience

It is easy to conclude that Quicknode provides the quickest response times for Solana globally, as well as the highest data reliability and stability. Achieving this feat requires more than just running the validator software. It takes innovation, similar to Solana, to meet all the requirements and maintain consistency. Quicknode continually develops powerful elements with exceptional capabilities that work alongside regular Solana RPC to provide maximum efficiency for developers. This is how Quicknode excels!

Lastly, the purpose of this ongoing exercise is to set and maintain high standards, ensuring that we are pushing the blockchain ecosystem forward. Being an infrastructure provider and building an ecosystem around it takes time, effort, and experience. Quantifying the quality of a service requires data-driven decision-making, not biased numbers or gimmicky keywords. Ultimately, these insights help us improve the industry and support builders, which is the end goal.

Appendix

• Live Data Dashboard

• Solana Block Derby Data Dump — GitHub Repository | SQL Queries | Data Dump (CSV)

• Justifying Quick in Quicknode: A Response Time Comparison of Blockchain Node Providers

• (Feb 1 2024) Quicknode Continues to Reinvest in its Solana Infra

Warum Quicknode?

Quicknode is an affordable, globally-distributed, multi-chain, multi-client, premium Web3 infrastructure provider, powering industry leaders like 1inch, MagicEden, Exodus, and Nansen, to name a few. Quicknode serves more than 7 billion API requests per day (at the time of writing this article). Here's why businesses choose Quicknode for Web3 infrastructure!

Moreover, Quicknode is not only a blockchain node provider for 18+ chains but also an ecosystem in itself with a full suite of blockchain APIs and products — Core API, Token API, NFT API, GraphQL API, QuickAlerts — and the first of its kind, blockchain API Marketplace.

We say, "Do your stuff —build — and let us take care of the rest!"