--- author: Sergen Uysal title: How to Validate Incoming Streams Webhook Messages tags: [All, Ethereum, EVM, Quicknode Product, Streams, Security, Python] description: In this guide, you will learn how to ensure the integrity and authenticity of incoming webhook messages by verifying HMAC signatures using Python. You will retrieve the necessary headers, prepare the payload, and execute a Python script to perform signature verification. --- _11 min read_ Streams Streams is available starting from the Free [plan](https://www.quicknode.com/pricing?utm_source=internal&utm_campaign=guides&utm_content=validating-incoming-streams-webhook-messages). For teams with unique requirements, we offer tailored datasets, dedicated support, and custom integrations. [Contact](https://www.quicknode.com/contact-us) our team for more information. One Stream, One Cost, Multiple Destinations Your [Quicknode Stream](https://www.quicknode.com/streams) can now deliver to multiple destinations simultaneously from a single pipeline. No duplicate streams, no config drift, and no additional charge per destination. Available destinations per stream vary by plan. [Learn more](https://www.quicknode.com/docs/streams/destinations#multiple-destinations). ## Overview Streaming blockchain data in real-time can often be daunting due to the complexity of infrastructure setup and the ongoing management required. Quicknode addresses this challenge with [Streams](https://www.quicknode.com/streams?utm_source=internal&utm_campaign=guides&utm_content=validating-incoming-streams-webhook-messages), a blockchain data streaming solution that allows you to easily stream both historical and real-time data directly to your applications or services. This guide will take you through the process of verifying HMAC signatures for incoming messages from [Streams](https://www.quicknode.com/streams?utm_source=internal&utm_campaign=guides&utm_content=validating-incoming-streams-webhook-messages) webhook services. Ensuring the integrity and authenticity of these messages is crucial for securing your applications against tampering and forgery. By the end of this guide, you'll be equipped with the knowledge to implement signature verification using Python effectively. ### What You Will Do - Implement webhook signature verification in Node.js, Python, or Go - Set up a server to handle incoming webhook messages - Verify signatures using HMAC SHA-256 - Process both compressed and uncompressed payloads - Test your verification implementation - Discuss best security practices to enhance the safety of your verification process ### What You Will Need - Your preferred language environment (Node.js, Python, or Go) - A code editor (e.g., VSCode) - [ngrok](https://ngrok.com/) installed - Your [Stream's](https://www.quicknode.com/signup?utm_source=internal&utm_campaign=guides&utm_content=validating-incoming-streams-webhook-messages) security token from the Quicknode dashboard - Language-specific requirements (see dependencies table below) **Node.js** | Dependency | Version | | --- | --- | | node.js | \\>=16.x | | express | ^4.18.2 | | body-parser | ^1.20.2 | | ngrok | ^3.0.0 | **Python** | Dependency | Version | | --- | --- | | python | \\>=3.7 | | flask | ^2.0.0 | | ngrok | ^3.0.0 | **Go** | Dependency | Version | | --- | --- | | go | \\>=1.16 | | ngrok | ^3.0.0 | ## Understanding Webhook Signatures When your server receives a webhook from Streams, it includes three critical headers that are used for verification: - **X-QN-Nonce**: A unique string that prevents replay attacks - **X-QN-Signature**: The HMAC signature you need to verify - **X-QN-Timestamp**: The timestamp when the message was signed The signature is created by combining these elements with the payload in a specific way, then applying an HMAC-SHA256 hash using your Stream's security token as the key. Now, we'll move onto the coding part of the guide and show you how to validate incoming webhook messages from Streams. ### Gzip Payloads and What to Verify Streams may send the webhook body with **`Content-Encoding: gzip`**. The HMAC is always computed over the **decoded (uncompressed) JSON payload**, the same UTF-8 string that was signed **before** compression, concatenated as **`nonce + timestamp + payload`**. You should **decode or decompress the body first**, then verify the signature using that string (not the gzip bytes on the wire). Node.js (Express) and automatic gzip decoding [`body-parser`](https://github.com/expressjs/body-parser) (including **`express.raw()`** and **`bodyParser.raw()`**) **automatically inflates** request bodies when **`Content-Encoding: gzip`**, by default (`inflate` is enabled). The raw bytes collected into **`req.body`** are therefore **already uncompressed**. Use **`req.body.toString('utf8')`** (or equivalent) as the payload for verification. The **`Content-Encoding: gzip`** header is **not** removed from the request, so it can still appear even though **`req.body`** holds decoded JSON. **Do not** run `gunzip` again on **`req.body`** in that situation, or verification helpers will fail (for example with an "incorrect header check" error from zlib). For a complete **Node.js / Express** sample you can clone and run, including environment-based secrets, raw body handling, and the gzip behavior above, see **[quiknode-labs/streams-webhook-validate-signature](https://github.com/quiknode-labs/streams-webhook-validate-signature)** on GitHub. In **Python** and **Go**, you typically read the request bytes and **explicitly** decompress when **`Content-Encoding`** is **`gzip`** before building the string for HMAC, as in the examples below. ## Stream Setup and Security Token First, we'll need a security token for signature verification. You can either: - **If you have an existing Stream**: Go to your [Stream's](https://dashboard.quicknode.com/streams) **Settings** tab and keep your security token handy (we'll come back to this later) - **If you need to create a Stream**: 1. Visit [TypedWebhook.tools](https://typedwebhook.tools) and copy the provided webhook URL 2. Go to the [Streams](https://dashboard.quicknode.com/streams) section on your Quicknode dashboard 3. Click **Create Stream** 4. Configure your Stream: - Select **Ethereum** as the blockchain and **Mainnet** as your network - Choose **Blocks** as your dataset - Paste the TypedWebhook.tools URL in the webhook destination settings 5. Create the Stream and copy your security token via the Settings tab **Note**: This TypedWebhook.tools URL is temporary until we show you later on in the guide how to run your own local server and ngrok to validate the incoming message. ## Setting Up Your Development Environment First, create a new project directory and set up your environment based on your chosen language: **Node.js** ```sh mkdir webhook-verification cd webhook-verification npm init -y npm install express body-parser ``` **Python** ```sh mkdir webhook-verification cd webhook-verification pip install flask ``` **Go** ```sh mkdir webhook-verification cd webhook-verification go mod init webhook-verification ``` ## Implementing the Verification Server Now, let's implement the verification server. We'll create a server that: - Listens for POST requests on a `/webhook` endpoint - Extracts the necessary headers (e.g., nonce, signature, timestamp) - Verifies the signature - Returns HTTP responses Choose your preferred language and create your server file within the appropriate directory (based on the language you chose). **Remember** to replace the `your_security_token_here` placeholder string with your actual security token. **Node.js** You can clone and run a complete **Express** implementation of this guide, including gzip handling and env-based secrets, from **[quiknode-labs/streams-webhook-validate-signature](https://github.com/quiknode-labs/streams-webhook-validate-signature)** on GitHub. ```javascript const express = require('express'); const crypto = require('crypto'); const bodyParser = require('body-parser'); const app = express(); const PORT = 9999; // Raw parser: when Content-Encoding is gzip, body-parser decodes gzip into req.body by default. // Verify HMAC using req.body.toString('utf8') — the uncompressed JSON Streams signed. app.use(bodyParser.raw({ type: '*/*', limit: '50mb' })); // Optional: only needed if you add other routes that expect JSON; not required for /webhook below. app.use(bodyParser.json()); function verifySignature(secretKey, payload, nonce, timestamp, givenSignature) { // First concatenate as strings const signatureData = nonce + timestamp + payload; // Convert to bytes const signatureBytes = Buffer.from(signatureData); // Create HMAC with secret key converted to bytes const hmac = crypto.createHmac('sha256', Buffer.from(secretKey)); hmac.update(signatureBytes); const computedSignature = hmac.digest('hex'); console.log('\nSignature Debug:'); console.log('Message components:'); console.log('- Nonce:', nonce); console.log('- Timestamp:', timestamp); console.log('- Payload first 100 chars:', payload.substring(0, 100)); console.log('\nSignatures:'); console.log('- Computed:', computedSignature); console.log('- Given:', givenSignature); return crypto.timingSafeEqual( Buffer.from(computedSignature, 'hex'), Buffer.from(givenSignature, 'hex') ); } app.post('/webhook', async (req, res) => { const secretKey = 'your_security_token_here'; const nonce = req.headers['x-qn-nonce']; const timestamp = req.headers['x-qn-timestamp']; const givenSignature = req.headers['x-qn-signature']; if (!nonce || !timestamp || !givenSignature) { console.error('Missing required headers'); return res.status(400).send('Missing required headers'); } try { // Payload for HMAC: decoded JSON string (body-parser already gunzipped if Content-Encoding was gzip). const payloadString = req.body.toString('utf8'); const isValid = verifySignature( secretKey, payloadString, nonce, timestamp, givenSignature ); if (isValid) { console.log('\n✅ Signature verified successfully'); return res.status(200).send('Webhook received and verified'); } else { console.log('\n❌ Signature verification failed'); return res.status(401).send('Invalid signature'); } } catch (error) { console.error('Error processing webhook:', error); return res.status(500).send('Error processing webhook'); } }); app.listen(PORT, () => { console.log(`Server running on port ${PORT}`); }); ``` **Python** ```python import hmac import hashlib import gzip from flask import Flask, request, jsonify import logging app = Flask(__name__) # Configure logging logging.basicConfig(level=logging.INFO) logger = logging.getLogger(__name__) # Replace this with your actual security token SECRET_KEY = "your_security_token_here" def verify_signature(secret_key, payload, nonce, timestamp, given_signature): message = nonce + timestamp + payload computed_signature = hmac.new( secret_key.encode(), message.encode(), hashlib.sha256 ).hexdigest() return hmac.compare_digest(computed_signature, given_signature) @app.route('/webhook', methods=['POST']) def handle_webhook(): nonce = request.headers.get('X-QN-Nonce') timestamp = request.headers.get('X-QN-Timestamp') given_signature = request.headers.get('X-QN-Signature') if not all([nonce, timestamp, given_signature]): logger.error("Missing required headers") return jsonify({"error": "Missing required headers"}), 400 # Get the raw payload raw_payload = request.get_data() # Check if the payload is gzip compressed if request.headers.get('Content-Encoding') == 'gzip': try: payload = gzip.decompress(raw_payload).decode('utf-8') except Exception as e: logger.error(f"Error decompressing payload: {str(e)}") return jsonify({"error": "Failed to decompress payload"}), 400 else: payload = raw_payload.decode('utf-8') try: is_valid = verify_signature(SECRET_KEY, payload, nonce, timestamp, given_signature) except Exception as e: logger.error(f"Error verifying signature: {str(e)}") return jsonify({"error": "Failed to verify signature"}), 500 if is_valid: logger.info("Received valid webhook") # Process the webhook payload here # For now, we'll just return a success message return jsonify({"message": "Webhook received and verified"}), 200 else: logger.warning("Received invalid webhook") return jsonify({"error": "Invalid signature"}), 401 if __name__ == '__main__': app.run(debug=True, port=5000) ``` **Go** ```go package main import ( "bytes" "compress/gzip" "crypto/hmac" "crypto/sha256" "encoding/hex" "errors" "fmt" "io" "log" "net/http" ) // handler for POST requests func postHandler(w http.ResponseWriter, r *http.Request) { // Ensure the method is POST if r.Method != http.MethodPost { http.Error(w, "Only POST method is allowed", http.StatusMethodNotAllowed) return } signature := r.Header.Get("X-QN-Signature") nonce := r.Header.Get("X-QN-Nonce") timestamp := r.Header.Get("X-QN-Timestamp") secretKey := "your_security_token_here" // Read the request body body, err := io.ReadAll(r.Body) if err != nil { http.Error(w, "Failed to read body", http.StatusInternalServerError) return } defer r.Body.Close() // Check if the body is encoded with gzip if r.Header.Get("Content-Encoding") == "gzip" { gzipReader, err := gzip.NewReader(bytes.NewReader(body)) if err != nil { http.Error(w, "Failed to create gzip reader", http.StatusInternalServerError) return } defer gzipReader.Close() decodedBody, err := io.ReadAll(gzipReader) if err != nil { http.Error(w, "Failed to read gzip body", http.StatusInternalServerError) return } body = decodedBody } err = VerifyHMAC(secretKey, nonce, timestamp, string(body), signature) if err != nil { fmt.Println(err) http.Error(w, "Invalid HMAC", http.StatusUnauthorized) return } fmt.Println("HMAC is valid") // Respond to the client w.WriteHeader(http.StatusOK) _, _ = w.Write([]byte("Received POST request")) } func main() { http.HandleFunc("/", postHandler) // Start the server on port 8080 port := ":8080" fmt.Printf("Server is listening on port %s\n", port) log.Fatal(http.ListenAndServe(port, nil)) } func VerifyHMAC(secretKey, nonce, timestamp, message, receivedHMAC string) error { // Combine nonce, timestamp, and message data := nonce + timestamp + message // Create a new HMAC using SHA-256 and the secret key h := hmac.New(sha256.New, []byte(secretKey)) h.Write([]byte(data)) // Compute the expected HMAC expectedHMAC := hex.EncodeToString(h.Sum(nil)) // Compare the received HMAC with the expected HMAC if !hmac.Equal([]byte(expectedHMAC), []byte(receivedHMAC)) { return errors.New("invalid HMAC: message integrity or authenticity check failed") } // HMAC is valid return nil } ``` ## Starting Your Server Start your verification server based on your implementation: **Node.js** ```sh node server.js ``` **Python** ```sh python verify_signature.py ``` **Go** ```sh go run main.go ``` ## Setting Up ngrok Then, create a tunnel to your local server (adjust the port based on your implementation): **Node.js** ```sh ngrok http 9999 ``` **Python** ```sh ngrok http 5000 ``` **Go** ```sh ngrok http 8080 ``` Keep the terminal window handy as we'll need the URL when updating our Stream's webhook destination. ## Validating Messages with Incoming Stream Data 1. First, copy the ngrok URL running in your terminal (e.g., [https://abc123.ngrok.io](https://abc123.ngrok.io)) 2. Then, go to your Stream's settings in the Quicknode dashboard. 3. Pause your Stream and update the webhook URL to your ngrok URL + `/webhook` (e.g., [https://abc123.ngrok.io/webhook](https://abc123.ngrok.io/webhook)) 4. Resume your Stream 5. Watch your server logs. For each block, you should see output similar to: **Node.js** ```sh Signature Debug: Message components: - Nonce: 02c6d2644296c8b830970891410825a3 - Timestamp: 1735613672 - Payload first 100 chars: {"data":[{"baseFeePerGas":"0xd557bf66","blobGasUsed":"0x60000","difficulty":"0x0","excessBlobGas":"0 Signatures: - Computed: 049827ff010a1c24cd21594d72e490fd43a48d9e69a5c18628788063665134cc - Given: 049827ff010a1c24cd21594d72e490fd43a48d9e69a5c18628788063665134cc ✅ Signature verified successfully ``` **Python** ```sh INFO:__main__:Received valid webhook INFO:werkzeug:127.0.0.1 - - [30/Dec/2024 21:53:01] "POST /webhook HTTP/1.1" 200 - ``` **Go** ```sh Server running on port :8080 HMAC is valid ``` If you see successful verification messages, congrats! Your server is properly verifying webhook signatures. ## Best Security Practices While implementing the HMAC signature verification, ensure the following practices to secure your application: - Always **keep the security token confidential** and securely stored, not hardcoded in publicly accessible areas of your code. - **Log all validation attempts**, both successful and failed, to aid in auditing and troubleshooting. - Implement **additional checks** like timestamp validation to prevent replay attacks. ## Conclusion Congratulations! You've successfully learned how to validate incoming Streams webhook messages by verifying HMAC signatures. This practice is vital for protecting your applications from external threats and ensuring data integrity. If you have questions, please [contact us](https://www.quicknode.com/contact-us) directly. If you have any ideas or suggestions, such as new destinations, features, metrics, or datasets, you want us to support. Also, stay up to date with the latest by following us on [Twitter](https://twitter.com/Quicknode) and joining our [Discord](https://discord.gg/quicknode) and [Telegram announcement channel](https://t.me/quicknodehq). #### We ❤️ Feedback! [Let us know](https://airtable.com/shrKKKP7O1Uw3ZcUB?prefill_Guide+Name=How%20to%20Validate%20Incoming%20Streams%20Webhook%20Messages) if you have any feedback or requests for new topics. We'd love to hear from you.