Forem: Emily

Handling Real-Time HK Stock Quotes Without Timeouts: A WebSocket Approach

Emily — Mon, 18 May 2026 21:30:34 +0000

If you’re building any kind of HK stock market tool, you’ve probably hit the wall where your HTTP requests start timing out—especially during the opening rush. In this post, I’ll walk through the exact steps I took to stabilize a brokerage advisory dashboard, moving from fragile polling to a robust WebSocket pipeline.

1. Understanding the Problem

Timeout errors usually stem from two places: the data provider’s gateway being overloaded (leading to actual network timeouts), and your local processing being too slow (causing back-pressure that starves the socket). When I profiled my own app, I realized that synchronous database writes inside the callback were delaying the event loop just enough to trigger downstream timeouts.

2. Stop Polling, Start Streaming

The first architectural change is to replace periodic HTTP pulls with a persistent WebSocket connection. Once established, the server pushes each tick in real time. For HK stocks, I rely on AllTick’s WebSocket feed—it accepts a subscription list and delivers a continuous stream, removing the need for constant reconnections.

3. Code Example: WebSocket Subscription in Python

Here’s a minimal working example that subscribes to Tencent and Alibaba HK shares:

import websocket
import json

def on_message(ws, message):
    data = json.loads(message)
    print(data)  # Received tick data; in production, push to a queue or DB

def on_open(ws):
    subscribe_data = {
        "action": "subscribe",
        "symbols": ["00700.HK", "09988.HK"]  # Symbols to subscribe to
    }
    ws.send(json.dumps(subscribe_data))

ws = websocket.WebSocketApp(
    "wss://api.alltick.co/ws/stock",
    on_message=on_message,
    on_open=on_open
)
ws.run_forever()

Keep the callback light; just parse and forward.

4. Batch Subscriptions and Throttle Requests

When subscribing to many instruments at once, split the list into chunks of 10–20 and insert a 50–100 ms pause between chunks. This prevents the initial burst from overwhelming the server’s TCP buffer and avoids throttling.

5. Implementing Reconnection with Exponential Backoff

Even WebSocket connections drop. I set up a heartbeat (ping every 30s, reconnect after two missed pongs). Reconnection attempts follow an exponential backoff: 1s, 2s, 4s, … max 30s. If only a single symbol fails, I re-subscribe just that one to avoid disturbing the rest.

6. Offload Processing with Async and Queues

Move any heavy work—indicator computation, database writes—out of the on_message thread. I use a queue.Queue and a consumer thread that flushes data in batches. Bulk inserts (every 100 ticks or 200ms) keep the database happy and eliminate write-induced timeouts.

7. Final Tips

Since moving to this WebSocket + queue architecture, the advisory dashboard I maintain hasn’t experienced a single client-visible timeout during trading hours. Start by swapping your transport, then optimize your processing, and your HK stock data pipeline will become orders of magnitude more reliable.

Dynamic Forex Pair Subscription in Python: Stop Spamming Your WebSocket Data Feed

Emily — Fri, 15 May 2026 05:21:22 +0000

Have you ever had your live trading bot kicked off the data server just when the market started moving? I certainly have. The culprit wasn't my trading strategy, but how I was shouting subscribe and unsubscribe commands at my WebSocket connection. Let me share how I built a polite, stateful client that keeps the data flowing cleanly.

When you're developing a high-frequency forex bot as a solo dev, you quickly realize that a real-time feed isn't a static resource. Your strategy constantly shifts focus, and your code must tell the server to add new currency pairs and drop old ones without reconnecting. If you get this wrong, you'll either flood yourself with useless ticks or get rate-limited into oblivion. Let's fix that.

The Problem: Stateless Requests to a Stateful Connection

We often treat a WebSocket like a REST endpoint, firing off messages whenever we feel like it. Imagine a breakout strategy that monitors EURUSD. The moment volatility spikes, it wants to also watch GBPUSD and USDJPY. A naive implementation does this:

# Don't do this at home!
if volatility_spikes:
    ws.send(json.dumps({"action": "subscribe", "symbols": ["GBPUSD"]}))
    ws.send(json.dumps({"action": "subscribe", "symbols": ["USDJPY"]}))

If that condition oscillates during a choppy market, you'll hammer the server with rapid, repeated subscription attempts. The platform's firewall will likely classify your script as misbehaving and drop the connection. Now you're disconnected during the exact volatility you wanted to trade.

The Solution: A Local Subscription Mirror

The trick is to keep a client-side record of exactly what's currently active. Let this local object be the gatekeeper. Any subscription request gets checked against it; any cancellation is only sent if the pair is truly active.

I use this pattern with the AllTick real-time API, but it works with any provider that supports dynamic subscription messages. Here’s a battle-tested code snippet:

import websocket
import json

# Message handler for incoming ticks
def on_message(ws, message):
    data = json.loads(message)
    print("Tick:", data)

ws = websocket.WebSocketApp("wss://api.alltick.co/realtime",
                            on_message=on_message)

# Our local state mirror
subscribed = {}

def safe_subscribe(symbols):
    # Filter out already active symbols
    new_list = [s for s in symbols if s not in subscribed]
    if new_list:
        ws.send(json.dumps({"action": "subscribe", "symbols": new_list}))
        for s in new_list:
            subscribed[s] = True

def safe_unsubscribe(symbols):
    # Filter out symbols we aren't watching
    active_list = [s for s in symbols if s in subscribed]
    if active_list:
        ws.send(json.dumps({"action": "unsubscribe", "symbols": active_list}))
        for s in active_list:
            subscribed.pop(s)

ws.run_forever()

With safe_subscribe and safe_unsubscribe, your strategy can be as noisy as it wants. The gateway absorbs the duplicates and only sends clean, minimal diffs to the server. No more disconnection panic.

Pro Tips: Batching and Data Handling

Batch your commands: If you need to add five pairs and remove three, compute the entire diff and send a single subscribe and a single unsubscribe message. One round trip, not eight.
Cache the latest price: Don't process ticks directly in the on_message callback. Update a global latest_price dictionary and have your strategy loop read it asynchronously.
Use a separate thread for I/O: Let a writer thread accumulate ticks and flush them to your database every 500ms. This prevents the GIL or I/O waits from slowing down your message parsing.

Managing WebSocket subscriptions feels like a tiny plumbing task, but doing it right is what separates a weekend prototype from a robust 24/7 trading system. How do you handle dynamic subscriptions in your projects? Let me know in the comments!

Stream US stock ticks in real time with Python and WebSocket

Emily — Thu, 14 May 2026 02:13:08 +0000

The problem with polling market data

If you’ve ever tried to fetch live US stock prices using REST APIs, you know the struggle: rate limits kick in when you need data the most, and the time between requests creates blind spots. For tick-by-tick analysis — like detecting volume spikes or order-flow imbalances — polling just doesn’t cut it.

Real-time push with WebSocket

WebSocket lets the server push every trade directly to you, eliminating polling delays and rate limits. I recently built a real-time tick feed for hot US stocks using Python and a WebSocket API. The data provider I used (AllTick) delivers each trade as a JSON object with symbol, price, and volume — super easy to parse.

Let’s see the code

Here’s the simplest version to get you started:

import websocket
import json

def on_message(ws, message):
    data = json.loads(message)
    # Print price and volume for every trade
    print(f"{data['symbol']} price: {data['price']} volume: {data['volume']}")

def on_open(ws):
    # Subscribe to a few popular tickers
    symbols = ["AAPL", "TSLA", "AMZN"]
    for symbol in symbols:
        ws.send(json.dumps({
            "action": "subscribe",
            "symbol": symbol
        }))

ws = websocket.WebSocketApp("wss://apis.alltick.co/ws/stock-tick",
                            on_message=on_message,
                            on_open=on_open)
ws.run_forever()

Run it, and you’ll see a live stream of every trade for Apple, Tesla, and Amazon. It’s that fast.

What to do with the data

Store it in Redis for ultra-fast access and real-time dashboards.
Archive to SQLite or PostgreSQL for backtesting.
Build a Streamlit dashboard to visualize volume leaders in real time.

Make it production-ready

Auto-reconnect: Handle on_close and reconnect with backoff.
Deduplication: Use the trade ID to avoid double-counting.
Async: Use asyncio or websockets library for non-blocking I/O.
Heartbeat: Keep the connection alive with ping/pong.

Final thoughts

Streaming real-time stock ticks is easier than most developers think. With a few lines of Python and a reliable WebSocket endpoint, you can turn your local machine into a powerful market monitoring station. I’ve been using this setup daily to feed our quant models, and it hasn’t missed a beat. Give it a try and see what insights you can uncover!

How to Handle Weekend Data Gaps in XAUUSD Backtesting Without Losing Your Mind

Emily — Fri, 08 May 2026 07:39:12 +0000

The bug that wasn’t a bug

Confession time: I spent a whole week debugging a gold strategy that kept showing a mysterious profit spike every Monday at 00:00. The logic was fine. The data seemed fine. Then I realized my realtime feed simply didn’t send any XAUUSD ticks over the weekend, and my backtester was treating 48 hours of no-data as “price stayed flat.” That flat stretch was making the Monday gap look like a tradeable trend, and my metrics were lying to me.

If you’re building a gold trading system, this is almost certainly affecting your backtests too. Let’s walk through why, and how to code around it.

The mechanics of backtest distortion

Most backtesting loops work like this: for each time step, if there’s no new price, keep the old one. Over a weekend, that gives you a long flat line connecting Friday’s closing price to Monday’s opening price. When the new tick finally arrives, the instantaneous jump is fed directly to your indicators.

This artificially inflates trend-following signals and deflates volatility estimates. The model sees a smooth entry into a gap, but the real market offers only a discontinuous fill. The result? Beautiful backtests that dissolve in live trading.

Which fix is worth the effort

Here’s a quick comparison based on what I’ve tried:

Method	Backtest Fidelity	Implementation Effort	Recommendation
Ignore weekends completely	Poor	Trivial	Not recommended
Fill with Friday’s closing price	Moderate	Easy	Barely acceptable
Replace with Monday’s opening price	Moderate	Easy	Average
Mark gap intervals as events	Good	Medium	Recommended
Incorporate external reference data	High	High	Use as needed

I strongly suggest going with explicit gap marking. It involves a bit more code, but it preserves the discontinuity and lets you analyze how the strategy behaves around gaps — essential intel for gold.

Sample code: separating weekend ticks from strategy signals

I’ll show you a minimal example using a WebSocket connection to the AllTick API. The key is a date check that routes weekend data away from the strategy.

import websocket
import json
from datetime import datetime

def on_message(ws, message):
    data = json.loads(message)
    timestamp = datetime.fromtimestamp(data['time'])
    # Weekend tick: store for gap analysis only, don't feed to strategy
    if timestamp.weekday() >= 5:
        print(f"Weekend data {data['price']} written to gap buffer")
        # Save to gap storage
        store_gap_data(data)
    else:
        # Regular weekday tick
        process_tick(data['symbol'], data['price'])

ws = websocket.WebSocketApp(
    "wss://apis.alltick.co/websocket-api/stock-websocket-interface-api/transaction-quote-subscription",
    on_message=on_message
)
ws.run_forever()

Write a small gap injection routine for your backtester that applies the Friday-close to Monday-open jump before the first tick of the week. Now your simulation matches reality.

Tips for honest gold backtests

Time axis should be partitioned by trading sessions, not calendar days.
Analyze gap PnL separately — if your edge comes purely from weekend jumps, you don’t have an edge.
Keep the exact same data processing path in production and research.

The fact that a gold API doesn’t push data on weekends is a guarantee of its accuracy. By explicitly modeling those gaps, you turn a hidden pitfall into a well-understood component of your system.

How to Build a Real‑Time Multi‑Currency Forex Volatility Dashboard with Python and WebSockets

Emily — Thu, 07 May 2026 02:31:28 +0000

As a part‑time forex trader and full‑time tinkerer, I wanted a dashboard that could show me, in real time, which currency pairs were suddenly becoming volatile. Standard charting platforms update volatility indicators at the close of each candle — too slow for my style of scalping. So I built my own tick‑by‑tick volatility monitor using Python, WebSockets, and ECharts. In this tutorial, I’ll walk you through the exact steps.

1. The Problem with Polling

I initially tried polling a REST API every second: fetch the latest price for a pair, store it in a list, compute standard deviation over the last 60 data points. It was simple, but the volatility curve was always lagging behind the real action. Polling captures only a tiny fraction of the tick stream, producing a heavily aliased signal. To get true micro‑volatility, you need to consume every tick as it happens. That‘s where WebSockets come in.

2. Setting Up the Real‑Time Data Stream

I chose a provider that offers a WebSocket API for forex ticks. With a single connection, you can subscribe to multiple instruments like EUR/USD, USD/JPY, GBP/USD. The setup is minimal — here’s a boilerplate Python script that prints each incoming tick:

import websocket
import json

def on_message(ws, message):
    data = json.loads(message)
    # print the symbol and the latest traded price
    print(f"{data['symbol']} current price: {data['price']}")

ws = websocket.WebSocketApp("wss://api.alltick.co/realtime", on_message=on_message)
ws.run_forever()

(Note: AllTick is one example of a real‑time WebSocket data service; you can use any similar API.)

3. Computing Real‑Time Volatility with a Sliding Window

Once ticks are flowing, the next step is to turn them into a volatility number. I use a time‑based sliding window of 60 seconds, implemented with a deque. For each new tick, I add it to the window, remove any data older than 60 seconds, and then calculate the sample standard deviation of the remaining prices. This provides a volatility value that updates on every single tick.

import collections
import statistics
import time

# maintain a sliding window of the last 60 seconds of prices
window = collections.deque()
window_seconds = 60

def add_tick(price):
    now = time.time()
    window.append((now, price))
    # clean up expired entries from the front of the window
    while window and window[0][0] < now - window_seconds:
        window.popleft()
    prices = [p for t, p in window]
    if len(prices) > 1:
        vol = statistics.stdev(prices)
        print(f"Real-time volatility: {vol}")

The key advantage: you‘re not waiting for a candle to close. Volatility starts climbing the millisecond buying pressure intensifies.

4. Building the Dashboard (Without Overloading the Browser)

I moved all computation to a Python backend. The frontend is a static HTML page using ECharts that receives volatility updates via WebSocket or Server‑Sent Events. This keeps the browser lightweight — it only has to render the lines. I also added a simple visual rule: if a pair’s volatility exceeds a threshold, its line turns red, giving an immediate visual signal.

5. Managing Multiple Pairs Concurrently

To handle many pairs cleanly, my backend maintains a dictionary where each key is a currency pair string, and the value is an object containing that pair‘s tick deque, the latest price, and current volatility. The global WebSocket callback demultiplexes incoming messages by symbol and routes them to the correct pair’s handler. This pattern scales well and keeps the codebase tidy.

Wrapping Up

Building this dashboard changed how I perceive the market. Real‑time tick volatility reveals market rhythm in a way that minute bars can‘t. The code above provides a solid starting point — once you have the pipeline, you can extend it with multi‑timeframe volatility comparisons, alarms, or even integrate the volatility feed into your own trading bot. If you’re into forex algo trading, I highly recommend giving a live tick dashboard a try.

Understanding Trading Instrument APIs: Fetching and Managing Symbols in Python

Emily — Wed, 07 Jan 2026 01:44:29 +0000

Building automated trading systems requires more than just real-time market data. Knowing your instruments—the symbols you can actually trade—is equally important. Trading instrument APIs (sometimes called symbol APIs) provide structured information about available instruments, including metadata and contract details. Handling this data efficiently can help prevent invalid orders and mismatched trades.
Fetching Instrument Lists
Most brokers and exchanges expose endpoints that return instruments or symbols. Typical fields include:
Symbol/ticker (e.g., EURUSD, AAPL)
Asset type (forex, stock, crypto, futures)
Exchange or market
Lot size or contract specifications
Here’s a quick Python example using requests and pandas:

import requests
import pandas as pd

url = https://api.example.com/instruments
response = requests.get(url).json()

df = pd.DataFrame(response["data"])
print(df.head())

This simple snippet fetches the instrument list and converts it into a pandas DataFrame, making it easy to filter, sort, or analyze.
Filtering and Searching Instruments
Large exchanges can have thousands of symbols. Filtering by asset type or market can streamline your workflow:

forex_symbols = df[df["type"] == "forex"]
print(forex_symbols)

You can also combine multiple filters, such as exchange and asset type, to quickly locate the instruments relevant to your strategy.
Keeping Your Symbol Database Up-to-Date
Instrument lists change frequently—new products are added, and others get delisted. Automating daily updates ensures your system never tries to trade an outdated symbol. Depending on your setup, you can:
Fetch the latest symbols at the start of each session
Maintain a local cache and update periodically
Both approaches have pros and cons, but the key is consistency and automation.
Why It Matters
Incorrect instrument data can cause errors in backtesting and live trading alike. By combining trading instrument APIs with market data APIs, you ensure your system knows both what to trade and where to trade it.
Discussion
How do you manage symbol updates in your trading systems? Do you prefer fetching live every session or maintaining a local cache? Sharing your workflow can help others avoid common pitfalls.