Forem: Shridhar G Vatharkar

Create a Market Profile Dashboard Using Python

Shridhar G Vatharkar — Tue, 24 Jun 2025 09:18:28 +0000

In this guide, you’ll learn how to build a Market Profile dashboard using Python. We’ll walk through how to use Streamlit, Plotly, and our Forex, CFD, and Crypto Data API to bring market data to life. Whether you're a trader, analyst, or developer, this tutorial will help you visualize key market structure components like Value Areas, Points of Control (POC), and TPO (Time Price Opportunity) Counts.

Here’s what you’ll learn:

How to retrieve minute-level market data using the TraderMade API
How to calculate Market Profile elements such as Value Areas and POC
How to display your analysis interactively using Streamlit and Plotly

What You’ll Need Before You Start

To follow along, you should have a basic understanding of Python, some familiarity with Market Profile concepts, and access to a TraderMade Forex API key.

Let’s Get Started

We’ve broken down this tutorial into clear, manageable steps to make it easy to follow. First, we’ll begin by setting up your environment.

Step 1: Setting Up Your Environment

Make sure the following tools and libraries are installed:

pip install streamlit plotly pandas numpy python-dateutil tradermade

Getting Started

Begin by importing the necessary libraries and setting up the API connection.

import streamlit as st
import plotly.graph_objects as go
import pandas as pd
import numpy as np
import datetime
from dateutil.relativedelta import relativedelta
import streamlit.components.v1 as components
import tradermade as tm

tm.set_rest_api_key('Your_API_Key')  # Replace with your TraderMade API Key

Retrieve Minute-Level Data Using the TraderMade API

In this step, we'll create a function to fetch the market data required for building our Market Profile.

def get_data_raw(currency_pair, start_time, end_time, interval='minute'):
    try:
        # Fetch data using TraderMade SDK
        data = tm.timeseries(
            currency=currency_pair,
            start=start_time,
            end=end_time,
            interval=interval,
            period=30,
            fields=["open", "high", "low", "close"]
        )
        # print(data)
        # Print the response to see its structure
        # print("API Response:", data)

        # Convert the response directly into a DataFrame
        df = pd.DataFrame(data)
        df['volume'] = df['close']
        # Convert the 'date' column to datetime if it exists
        df = df.replace(0, np.nan)
        df = df.dropna()
        df.set_index("date", inplace=True)
        df = np.round(df, 4)
        if 'date' in df.columns:
            df['date'] = pd.to_datetime(df['date'])
            df.set_index('date', inplace=True)

        # Sort the DataFrame by index
        df.sort_index(inplace=True)
        return df


    except KeyError as e:
        print(f"KeyError: {e}")
        return pd.DataFrame()  # Return an empty DataFrame in case of an error

    except Exception as e:
        print(f"An error occurred: {e}")
        return pd.DataFrame()  # Return an empty DataFrame in case of a general error

def get_data_MP(g, p, f, F, cur, st, ed):
    df = get_data_raw(cur, st, ed)
    return df

Create Helper Functions

We’ll start by defining a function that identifies the most recent working day—this helps us skip non-trading days like weekends.

def last_work_date(date, format):
    if datetime.datetime.strptime(date, "%Y-%m-%d").weekday() == 6:
        return (datetime.datetime.strptime(date, "%Y-%m-%d")-relativedelta(days=2)).strftime(format)
    elif datetime.datetime.strptime(date, "%Y-%m-%d").weekday() == 0:
        return (datetime.datetime.strptime(date, "%Y-%m-%d")-relativedelta(days=3)).strftime(format)
    else:
        return (datetime.datetime.strptime(date, "%Y-%m-%d")-relativedelta(days=1)).strftime(format)

Next, we'll define two additional functions to set the desired level of detail for our TPO and Market Profile views.

def get_rd(currency):
    od_list = ["UKOIL", "OIL", "XAGUSD","EVAUST","LUNUST",'LTCUST','XMRUST']
    od2_list = ["JPY"]
    od3_list = ["AAPL", "AMZN", "NFLX", "TSLA", "GOOGL", "BABA", "TWTR", "BAC", "BIDU", 
    'XAUUSD','SOLUST','BNBUST','BCHUST','DSHUST','EGLUST']
    od4_list = ["UK100", "SPX500","FRA40", "GER30","JPN225","NAS100","USA30", "HKG33", 
    "AUS200","BTC","BTCUST",'ETHUSD',"ETHUST"]
    cfd_list = ["EURNOK", "EURSEK", "USDSEK","USDNOK","NEOUST","ETCUST","DOTUST", "UNIUST"]
    if currency in od_list or currency[3:6] in od2_list:
        ad = .01
        rd = 2
    elif currency in od3_list:
        ad = 0.1
        rd = 1
    elif currency in od4_list:
        ad = 2
        rd = 0
    elif currency in cfd_list:
        ad = .001
        rd = 3
    else:
        ad = 0.0001
        rd = 4
    return rd, ad

def get_ad(ad, max, min):
    if (max-min) > ad*2000:
        return ad*50
    if (max-min) > ad*1000:
        return ad*20
    if (max-min) > ad*300:
        return ad*5
    elif (max-min) > ad*100:
        return ad*2
    else:
        return ad

Compute TPO and Value Area

The first function, midmax_idx, helps identify the index of the highest value in an array that’s closest to its midpoint. The second function, calculate_value_area, builds a value area around the Point of Control (POC) within the market profile (mp).

It does this by accumulating TPO counts from price levels near the POC until the sum reaches a specified target volume (target_vol). The function then returns the price range—from min_idx to max_idx—that defines the value area. Note that this refers to TPO volume, which differs from actual trading volume.

def midmax_idx(array):
    if len(array) == 0:
        return None

    # Find candidate maxima
    # maxima_idxs = np.argwhere(array.to_numpy() == np.amax(array.to_numpy()))[:, 0]
    maxima_idxs = np.argwhere(array == np.amax(array))[:,0]
    if len(maxima_idxs) == 1:
        return maxima_idxs[0]
    elif len(maxima_idxs) <= 1:
        return None

    # Find the distances from the midpoint to find
    # the maxima with the least distance
    midpoint = len(array) / 2
    v_norm = np.vectorize(np.linalg.norm)
    maximum_idx = np.argmin(v_norm(maxima_idxs - midpoint))
    return maxima_idxs[maximum_idx]

def calculate_value_area(poc_volume, target_vol, poc_idx, mp):
    min_idx = poc_idx
    max_idx = poc_idx
    while poc_volume < target_vol:
        last_min = min_idx
        last_max = max_idx

        next_min_idx = np.clip(min_idx - 1, 0, len(mp) - 1)
        next_max_idx = np.clip(max_idx + 1, 0, len(mp) - 1)

        low_volume = mp.iloc[next_min_idx].vol if next_min_idx != last_min else None
        high_volume = mp.iloc[next_max_idx].vol if next_max_idx != last_max else None

        if not high_volume or (low_volume and low_volume > high_volume):
            poc_volume += low_volume
            min_idx = next_min_idx
        elif not low_volume or (high_volume and low_volume <= high_volume):
            poc_volume += high_volume
            max_idx = next_max_idx
        else:
            break
    return mp.iloc[min_idx].value, mp.iloc[max_idx].value

mp_dict = {"0000":"A","0030":"B","0100":"C","0130":"D","0200":"E","0230":"F","0300":"G","0330":"H","0400":"I","0430":"J","0500":"K","0530":"L","0600":"M","0630":"N","0700":"O","0730":"P","0800":"Q","0830":"R","0900":"S","0930":"T","1000":"U","1030":"V","1100":"W","1130":"X","1200":"a","1230":"b","1300":"c","1330":"d","1400":"e","1430":"f","1500":"g","1530":"h","1600":"i","1630":"j","1700":"k","1730":"l","1800":"m","1830":"n","1900":"o","1930":"p","2000":"q","2030":"r","2100":"s","2130":"t","2200":"u","2230":"v","2300":"w","2330":"x",}

Bringing It All Together

To wrap things up, we’ll use the following function to generate the complete Market Profile.

def cal_mar_pro(currency, study, freq, period, mode, fp, mp_st, mp_ed, date):
    st = datetime.datetime.strptime(mp_st, "%Y-%m-%d-%H:%M")
    print(currency, mp_st, mp_ed)
    # try:
    rf = get_data_MP("M", str(freq), "%Y-%m-%d-%H:%M" , "%Y-%m-%d-%H:%M",currency, mp_st, mp_ed)
    hf = rf.copy()
    last_price = rf.iloc[-1]["close"]

    rf = rf[["high","low"]]
    rf.index = pd.to_datetime(rf.index)
    #rf = rf[:96]
    rd, ad = get_rd(currency)
    max = round(rf.high.max(), rd)
    min = round(rf.low.min(), rd)
    ad = get_ad(ad, max, min)
    try:
      x_data = np.round(np.arange(min, max, ad).tolist(), rd)
    except:
      print("MP loop failed", currency)
    # x_data = x_data[::-1]
    y_data= []
    z_data = []
    y_data1= []
    z_data1 = []
    tocount1 = 0
    for item in x_data:
        alpha = ""
        alpha1 = ""
        alpha2 = ""
        for i in range(len(rf)):
            if rf.index[i] < date:
                if round(rf.iloc[i]["high"], rd) >= item >= round(rf.iloc[i]["low"], rd):
                    alpha += mp_dict[rf.index[i].strftime("%H%M")]
            elif rf.index[i] >= date:
                if round(rf.iloc[i]["high"], rd) >= item >= round(rf.iloc[i]["low"], rd):
                    alpha1 += mp_dict[rf.index[i].strftime("%H%M")]
        tocount1 += len(alpha1)
        y_data.append(len(alpha))
        y_data1.append(len(alpha1))
        z_data.append(alpha)
        z_data1.append(alpha1)
    # y_data = y_data[::-1]
    # y_data1 = y_data1[::-1]
    mp = pd.DataFrame([x_data, y_data1]).T
    mp = mp[::-1]
    mp = mp.rename(columns={0:"value",1:"vol"})
    #poc_idx = midmax_idx(mp.vol)
    poc_idx = midmax_idx(mp.vol.values)
    poc_vol = mp.iloc[poc_idx].vol
    poc_price = mp.iloc[poc_idx].value
    target_vol = 0.7*tocount1


    value_high,value_low = calculate_value_area(poc_vol, target_vol, poc_idx, mp)
    print("Value area",tocount1, 0.7*tocount1)


    return x_data, y_data,y_data1, z_data, z_data1, value_high, value_low, poc_price, last_price, hf

Now that we’ve set up the Market Profile data, it’s time to build the dashboard.

Streamlit Dashboard

We’ll structure the Streamlit app into two main sections: the sidebar and the main display area. The sidebar will handle user inputs and configuration options to make the dashboard interactive, while the main area will present the Market Profile charts and tables.

Sidebar

Let’s start by setting up the sidebar—it’s straightforward and intuitive.

# Streamlit code to take input
st.set_page_config(layout="wide")
st.title("Market Profile Dashboard")

# currency = st.sidebar.text_input("Enter Currency Pair (e.g., EURUSD):", "EURUSD")
category = st.sidebar.selectbox("Select Category", ["CFD", "Forex"], index=["CFD", "Forex"].index(st.session_state.category))

item_list = tm.cfd_list() if category == "CFD" else ["EURUSD", "GBPUSD", "AUDUSD", "USDCAD", "EURNOK", "USDJPY", "EURGBP", "BTCUSD", "USDCHF", "NZDUSD", "USDINR", "USDZAR", "ETHUSD", "EURSEK"]

# Ensure item_list is a list and contains the default currency
if isinstance(item_list, list) and st.session_state.currency in item_list:
    index = item_list.index(st.session_state.currency)
else:
    index = 0  # Default to the first item if the currency is not in the list

currency = st.sidebar.selectbox("Select an Item", item_list, index=index)
study = st.sidebar.selectbox("Select Study Type:", ["MP"])
date = st.sidebar.date_input("Select Date")

# Subtract one day using relativedelta to get the previous day at 00:00 hours
mp_st = last_work_date(date.strftime('%Y-%m-%d'), '%Y-%m-%d-00:00')
if date != datetime.datetime.now().date():
   mp_ed = date.strftime('%Y-%m-%d-23:59')
else:
   mp_ed = datetime.datetime.now().strftime('%Y-%m-%d-%H:%M')    

# Convert to string format
# mp_st = previous_day.strftime("%Y-%m-%d-%H:%M")
freq = st.sidebar.selectbox("Select Minutes per period:", [30])
period = st.sidebar.selectbox("Select periods:", [24])
mode = st.sidebar.selectbox("Select Mode:", ["tpo"])
fp = int((60/int(freq))*period)

if date.weekday() in [5, 6]:  # 5 = Saturday, 6 = Sunday
   st.warning("Please select a weekday (Monday to Friday). Weekends are not allowed.")
   st.stop()  # Stop the app execution if the date is a weekend

# Check if the button has been clicked using session state
if "button_clicked" not in st.session_state:
   st.session_state.button_clicked = False

# Create the Market Profile chart only when the button is clicked
if st.sidebar.button("Get Market Profile"):
   st.session_state.button_clicked = True

Visualize with Streamlit & Plotly

Before we render the charts and TPO Profile, we’ll check if the user has clicked the button. Once confirmed, we’ll go ahead and display the visual output.

if st.session_state.button_clicked:
  #Get Market profile
  x_data, y_data,y_data1, z_data, z_data1, value_high, value_low, poc_price, last_price, hf = cal_mar_pro(currency, study, freq, period, mode, fp, mp_st,mp_ed, date)

  # Separate data for the previous day and current day
  previous_day_prices = x_data
  current_day_prices = x_data
  previous_day_counts = y_data
  current_day_counts = y_data1

  # Create the Market Profile chart for previous day
  candlestick = go.Candlestick(
      x=hf.index,
      open=hf['open'],
      high=hf['high'],
      low=hf['low'],
      close=hf['close'],
      name=f'{currency} Candlestick Chart',
  )
  tpo_dict = {}
  # Map index to date, using '00:00' hours formatting if necessary
  for i in range(len(hf.index)):
      tpo_dict[i] = hf.index[i]

  previous_day_dates = []
  current_day_dates = [] 

  for i in previous_day_counts:
      previous_day_dates.append(tpo_dict[i])
  for i in current_day_counts:
      current_day_dates.append(tpo_dict[i])

  # Create a vertical bar chart using the same x-axis (dates) and y-axis (volume)
  bar_chart = go.Bar(
      x=previous_day_dates,  # Use the same dates for x-axis
      y=previous_day_prices,  # Volume data for y-axis
      orientation='h',
      name="TPO Count previous day",
      marker=dict(
          color='rgba(50, 171, 96, 0.6)',
          line=dict(color='rgba(50, 171, 96, 1.0)', width=1)
      ),
      opacity=0.5,  # Adjust opacity to make both charts visible
      hoverinfo='skip'
  )

  bar_chart2 = go.Bar(
          x=current_day_dates,  # Use the same dates for x-axis
          y=current_day_prices,  # Volume data for y-axis
          orientation='h',
          name=f"TPO Count {date}",
          marker=dict(
          color='rgba(100, 149, 237, 0.6)',
          line=dict(color='rgba(100, 149, 237, 1.0)', width=1),
      ),
      opacity=0.5,  # Adjust opacity to make both charts visible
      hoverinfo='skip'
  )

  # Initialize the figure and add both traces
  fig1 = go.Figure()
  fig1.add_trace(candlestick)
  fig1.add_trace(bar_chart)
  fig1.add_trace(bar_chart2)
  print(y_data[0],y_data[-1:], poc_price)
  num_ticks = 6  # Number of ticks to display
  tick_indices = np.linspace(0, len(hf.index) - 1, num_ticks, dtype=int)
  tickvals = [hf.index[i] for i in tick_indices]
  ticktext = [hf.index[i] for i in tick_indices]

  # Update x-axis to show fewer date labels
  fig1.update_xaxes(
      tickvals=tickvals,  # Specify which dates to show
      ticktext=ticktext,  # Specify the labels for those dates
      tickangle=0,  # Keep the tick labels horizontal
      title_text='Date',
      showgrid=True,
  )
  fig1.update_yaxes(
      showgrid=True,
  )

  # Update layout for the combined chart
  fig1.update_layout(
      title=f'{currency} Candlestick Chart with TPO Bars',
      xaxis_title='Date/TPO Count',
      yaxis_title='Price',
      yaxis=dict(range=[x_data[0], x_data[-1]]),  # Reverse the y-axis
      height=600,
      xaxis=dict(
          type='category',  # Use 'category' type to skip missing dates
          categoryorder='category ascending',  # Order categories in ascending order
          showgrid=True,
      ),
      xaxis_rangeslider_visible=False  # Hide the range slider
  )

  # Create the Market Profile chart for current day
  fig2 = go.Figure()
  fig2.add_trace(go.Bar(
      x=previous_day_counts,  # Count on the x-axis
      y=previous_day_prices,  # Price levels on the y-axis
      orientation='h',
      name=f"TPO Count Previous day",
      marker=dict(
          color='rgba(50, 171, 96, 0.6)',
          line=dict(color='rgba(50, 171, 96, 1.0)', width=1)
      ),
      opacity=0.5
  ))

  fig2.add_trace(go.Bar(
      x=current_day_counts,  # Count on the x-axis
      y=current_day_prices,  # Price levels on the y-axis
      orientation='h',
      name=f"TPO Count {date}",
      marker=dict(
          color='rgba(100, 149, 237, 0.6)',
          line=dict(color='rgba(100, 149, 237, 1.0)', width=1),
      ),
  ))
  fig2.update_xaxes(
      showgrid=True,
  )
  fig2.update_yaxes(
      showgrid=True,
  )
  fig2.update_layout(
      title=f"{currency} Two-day Market Profile",
      xaxis_title="TPO Count",
      yaxis_title="Price Levels",
      template="plotly_white",
      height=600,
      width=600
  )

  # Create two columns to display charts side by side
  col1, col2 = st.columns([1, 1])

  with col1:
      st.plotly_chart(fig1, use_container_width=True)  # Adjusts the chart to fit the column width

  with col2:
      st.plotly_chart(fig2, use_container_width=True)

  style = """
  <style>
  table {
      width: 100%;
      border-collapse: collapse;
      font-family: 'Courier New', Courier, monospace;  /* Monospaced font */
  }
  th, td {
      border: 1px solid #ddd;
      padding: 8px;
      text-align: left;
      white-space: nowrap;  /* Ensure no text wrapping */
      font-weight: bold;
  }
  th {
      background-color: #f2f2f2;
      font-weight: bold;
  }
  tr:nth-child(even) {
      background-color: #f9f9f9;  /* Light gray background for even rows */
  }
  tr:hover {
      background-color: #f1f1f1;  /* Highlight row on hover */
  }
  </style>"""

  html_table = style
  html_table += "<table style='width:100%; border: 1px solid black; font-size: 12px; border-collapse: collapse;font-family: 'Courier New', Courier, monospace;'>"
  html_table += "<tr><th style='border: 1px solid black; padding: 2px;'>Prices</th>"
  html_table += "<th style='border: 1px solid black;padding: 2px;'>Previous Day</th>"
  html_table += f"<th style='border: 1px solid black;padding: 2px;'>{date}</th></tr>"

  # Populate the table rows
  for x, z, z1 in zip(reversed(x_data), reversed(z_data), reversed(z_data1)):
      if x in [value_low, poc_price, value_high]:
          html_table += f"<tr><td style='color:#FF6961; border: 1px solid black; padding: 0px;'>{x}</td>"
          html_table += f"<td style='color:#FF6961; border: 1px solid black; padding: 0px;'>{z}</td>"
          html_table += f"<td style='color:#FF6961; border: 1px solid black; padding: 0px;'>{z1}</td></tr>"
      else:       
          html_table += f"<tr><td style='border: 1px solid black; padding: 0px;'>{x}</td>"
          html_table += f"<td style='border: 1px solid black; padding: 0px;'>{z}</td>"
          html_table += f"<td style='border: 1px solid black; padding: 0px;'>{z1}</td></tr>"

  html_table += "</table>"

  html_table2 = style
  html_table2 += f"""
  <table style='width:100%; border: 1px solid black; border-collapse: collapse; font-size: 10px;'>
      <tr>
          <th style='border: 1px solid black; padding: 2px;'>Letter</th>
          <th style='border: 1px solid black; padding: 2px;'>Period</th>
      </tr>
  """

  # Populate the second table rows in reverse order or different formatting
  for letter, period in mp_dict.items():
      html_table2 += f"""
      <tr>
          <td style='border: 1px solid black; padding: 2px;'>{letter}</td>
          <td style='border: 1px solid black; padding: 2px;'>{period}</td>
      </tr>
      """

  html_table2 += f"</table>"

  html_table3 = style
  html_table3 += f"""
  <table style='width:100%; border: 1px solid black; border-collapse: collapse; font-size: 10px;'>
      <tr>
          <th style='border: 1px solid black; padding: 2px;'>Key Prices</th>
          <th style='border: 1px solid black; padding: 2px;'>Values</th>
      </tr>
  """

  html_table3 += f"""
      <tr>
          <td style='border: 1px solid black; padding: 2px;'>Value Area High</td>
          <td style='border: 1px solid black; padding: 2px;'>{value_high}</td>
      </tr>
      <tr>
          <td style='border: 1px solid black; padding: 2px;'>Point of Control (POC)</td>
          <td style='border: 1px solid black; padding: 2px;'>{poc_price}</td>
      </tr>
      <tr>
          <td style='border: 1px solid black; padding: 2px;'>Value Area Low</td>
          <td style='border: 1px solid black; padding: 2px;'>{value_low}</td>
      </tr>
      </table>
      """
  col3, col4 = st.columns([2, 2])
  with col1:
      components.html(html_table, height=2000, scrolling=True)
  with col2:
      components.html(html_table3,height=90, scrolling=True)
      components.html(html_table2, height=1200, scrolling=True)

The final section of the code is somewhat lengthy, but it primarily focuses on displaying the data using HTML tables and Plotly charts. If you're interested in customizing or learning more, you can explore Plotly and HTML documentation. For now, feel free to copy and paste this part as is.

Wrapping Up

You now have a fully functional and interactive Market Profile dashboard that works with Forex, CFD, and Crypto data. In just a few minutes, you can analyze different currency pairs, timeframes, and market structures.

To view the complete code, head over to our GitHub Market Profile examples page.

You can also enhance your dashboard by:

Experimenting with different time intervals, such as hourly or daily
Incorporating Volume Profile analysis
Adding alerts or triggers based on your trading strategies

Please go through the original tutorial on our website:
Build a Market Profile Dashboard with Python

Kotlin Guide for Retrieving Forex Market Data

Shridhar G Vatharkar — Thu, 12 Jun 2025 13:54:08 +0000

In today’s interconnected financial landscape, currency exchange rates have a significant impact on the global economy. For software developers, having access to reliable and real-time foreign exchange (forex) data is crucial for creating robust financial tools and applications. TraderMade offers a robust API that delivers real-time market data, historical exchange rates, and currency conversion information for a diverse range of currency pairs.

This Kotlin-based tutorial will guide you through using the TraderMade API with the Kotlin programming language. It covers the entire process—from setting up your Kotlin development environment to retrieving and displaying forex data from the API.

API Key

To start using the TraderMade API, create an account, go to the API section of the dashboard, select a suitable plan, and copy your unique API key for integration.

Let’s Begin Coding

To ensure clarity, we’ve broken down this Kotlin tutorial into manageable steps. Let's start by setting up the development environment.

Environment Setup

The following essential libraries are imported at the beginning of the code:

1. Retrofit – This library facilitates HTTP communication with REST APIs. It streamlines the process by enabling you to define API endpoints through Kotlin interfaces.

2. GsonConverterFactory – Used for converting API JSON responses into Kotlin data classes. This factory uses the popular Gson library to handle the JSON parsing.

3. OkHttpClient – Acts as the underlying HTTP client for Retrofit. It provides customization options for your network requests, such as timeout settings and interceptors.

4. Logger – This utility records log messages during runtime, making it easier to debug and trace the application’s behavior.

import retrofit2.Retrofit
import retrofit2.converter.gson.GsonConverterFactory
import retrofit2.Call
import retrofit2.http.GET
import retrofit2.http.Query
import okhttp3.OkHttpClient
import java.util.concurrent.TimeUnit
import java.util.logging.Logger

Defining Data Models

In this step, we create two Kotlin data classes that mirror the structure of the API response:

1. LiveCurrencyResponse – This class captures the complete response returned by the API. It includes a list of Quote objects, each representing a real-time forex quote.

2. Quote – This class contains the detailed attributes for each currency rate, including:

a) Ask – The selling price at which the currency is offered.

b) Bid – The buying price that a trader is willing to pay.

c) Mid – The average of the bid and ask prices, representing the mid-market rate.

d) base_currency and quote_currency – Indicate the pair of currencies being exchanged (e.g., USD as the base and EUR as the quote).

data class LiveCurrencyResponse(val quotes: List<Quote>)
data class Quote(val ask: Double, val bid: Double, val mid: Double, val base_currency: String, val quote_currency: String)

Creating the API Interface

This interface outlines the API endpoints and the methods used to interact with them:

1. @get("live") – This annotation indicates that the request is a GET call directed to the API’s “live” endpoint.

2. getLiveCurrencyData – This method defines the necessary parameters for the GET request:

a) apiKey – Your personal API key used for authenticating requests.

b) currencyPair – The specific currency pair you want data for (e.g., "USDEUR").

3. The method returns a Call object, which represents an asynchronous request. When executed, it retrieves the response from the API.

interface LiveCurrencyApiService {
    @GET("live")
    fun getLiveCurrencyData(@Query("api_key") apiKey: String, @Query("currency") currencyPair: String): Call<LiveCurrencyResponse>
}

Setting Up the Main Function

The main function begins by creating an instance of the Logger to enable logging. Next, it defines the API key and specifies the currency pair to be tracked—here, USD to EUR. This configuration is essential, as these values are required to perform the API call.

fun main() {
    val logger = Logger.getLogger("TraderMadeLogger")
    // Replace your API key
    val apiKey = "API_KEY"
    val baseCurrency = "USD"
    val quoteCurrency = "EUR"

Configuring OkHttpClient

In this step, we set up the OkHttpClient with specific timeout configurations:

1. Connect Timeout – Specifies the maximum time the client will wait while attempting to establish a connection.

2. Write Timeout – Specifies the maximum duration allowed for sending data to the server.

3. Read Timeout – Sets the limit for how long the client will wait to receive a response from the server.

These configurations help prevent the application from freezing and allow it to handle network delays or failures more effectively.

// OkHttpClient setup
    val client = OkHttpClient.Builder()
        .connectTimeout(10, TimeUnit.SECONDS)
        .writeTimeout(10, TimeUnit.SECONDS)
        .readTimeout(30, TimeUnit.SECONDS)
        .build()

Configuring Retrofit

Next, we set up Retrofit:

1) baseUrl: This is the base URL of the API you're working with. In this case, it's the TraderMade API. Note that this URL directs you to the endpoint where you can access live currency data; however, you can modify the URL to access other services, such as currency conversions or forex historical data.

2) Client: The OkHttpClient instance was configured earlier.

3) addConverterFactory: We add the GsonConverterFactory to handle JSON-to-Kotlin object conversion.

// Retrofit setup
    val retrofit = Retrofit.Builder()
        .baseUrl("https://marketdata.tradermade.com/api/v1/")
        .client(client)
        .addConverterFactory(GsonConverterFactory.create())
        .build()

Making the API Request

At this stage. We need to perform an API call to retrieve data:

1. Create – An instance of LiveCurrencyApiService is initialized using the configured Retrofit object.

2. getLiveCurrencyData – This method is invoked with the API key and desired currency pair as arguments. It returns a Call object representing the pending request.

3. Execute – The execute function sends a synchronous request to the API. The Logger is used to monitor and log the request's progress for better visibility and debugging.

// Fetch live currency data
    val liveCurrencyService = retrofit.create(LiveCurrencyApiService::class.java)
    val liveCurrencyCall = liveCurrencyService.getLiveCurrencyData(apiKey, "$baseCurrency$quoteCurrency")
    logger.info("Starting live currency request...")
    val liveCurrencyResponse = liveCurrencyCall.execute()
    logger.info("Live currency request finished.")

Handling the Response

Once the API call completes, the response is processed as follows:

1. isSuccessful – Verifies whether the request was completed successfully.

2. Quote – If the response is valid, the first Quote object from the returned list is retrieved.

3. resultText – The details of the quote, including ask, bid, and mid prices, are displayed.

4. Error Handling – If no quote is available or the request fails, informative messages are printed to indicate the issue.

if (liveCurrencyResponse.isSuccessful) {
        val quote = liveCurrencyResponse.body()?.quotes?.firstOrNull()
        if (quote != null) {
            val resultText = "Ask: ${quote.ask}\nBid: ${quote.bid}\nMid: ${quote.mid}"
            println(resultText)
        } else {
            println("No quotes found.")
        }
    } else {
        println("Live currency request failed with code: ${liveCurrencyResponse.code()}")
    }

Cleaning Up

To ensure efficient resource management and avoid memory leaks, the OkHttpClient is properly closed:

1. shutdown – Terminates the executor service responsible for handling network operations.

2. Evict All – Clears all unused or idle connections from the connection pool.

3. Logger – Logs the end of the program, indicating that execution has completed.

client.dispatcher.executorService.shutdown()
    client.connectionPool.evictAll()
    logger.info("Program finished.")
}

Note
In this Kotlin tutorial, we demonstrated how to work with the live currency data endpoint using the base URL https://marketdata.tradermade.com/api/v1/. TraderMade, however, provides various other endpoints that can be accessed by adjusting the URL. For instance, you can retrieve historical exchange rates or perform currency conversions by modifying the endpoint path accordingly. Check TraderMade's Data Documentation to explore API endpoints and other details.

Compiling the Kotlin Code

To convert your Kotlin source file (Realtime.kt) into bytecode executable by the Java Virtual Machine (JVM), use the following command:

1. kotlinc – This is the Kotlin compiler that transforms your Kotlin code into Java-compatible bytecode.

2. -cp – Short for "classpath," this option tells the compiler where to locate any external libraries your project depends on. Here, it includes multiple JAR files stored in the libs/ directory.

3. -d out – Specifies the output directory for the compiled bytecode. The compiled files will be saved to the out folder.

4. src/Realtime.kt – This indicates the file path to your Kotlin source code, which in this case is located in the src directory.

kotlinc -cp "libs/okio-jvm-3.9.0.jar;libs/retrofit-2.11.0.jar;libs/converter-gson-2.11.0.jar;libs/okhttp-4.12.0.jar;libs/okhttp-urlconnection-4.12.0.jar;libs/gson-2.11.0.jar;libs/annotations-24.1.0.jar;libs/kotlin-stdlib-2.0.0.jar" -d out src/Realtime.kt

Running the Compiled Kotlin Code

To execute the compiled Kotlin application, use the following command, which runs the program through the Java Virtual Machine (JVM):

1. Java – This command launches the Java Virtual Machine (JVM) and runs the specified class.

2. -cp "out; libs/..." – The -cp option sets the classpath, pointing to both the out directory (where compiled classes are stored) and the libs/ directory (containing required external libraries), so the JVM knows where to locate all dependencies.

3. com.api.RealtimeKt – This refers to the fully qualified name of the class containing the main function. Kotlin appends Kt to the class name when compiling, so Realtime.kt becomes RealtimeKt.

java -cp "out;libs/okio-jvm-3.9.0.jar;libs/retrofit-2.11.0.jar;libs/converter-gson-2.11.0.jar;libs/okhttp-4.12.0.jar;libs/okhttp-urlconnection-4.12.0.jar;libs/gson-2.11.0.jar;libs/annotations-24.1.0.jar;libs/kotlin-stdlib-2.0.0.jar" com.api.RealtimeKt

Putting It All Together

These commands work in sequence: the first compiles your Kotlin source code into JVM-compatible bytecode while linking the necessary JAR files, and the second runs the compiled program, utilizing those same dependencies at runtime.

Output

Below is the resulting output from running the program.

Aug 09, 2024 4:04:10 PM com.api.RealtimeKt main
INFO: Starting live currency request...
Aug 09, 2024 4:04:12 PM com.api.RealtimeKt main
INFO: Live currency request finished.
Ask: 0.9155665
Bid: 0.9155581
Mid: 0.9155581
Aug 09, 2024 4:04:12 PM com.api.RealtimeKt main
INFO: Program finished.

Conclusion

You’ve now completed the journey of using Kotlin to connect with the TraderMade API. This tutorial has provided you with the essential skills to access and utilize live forex data—a vital capability for developing modern financial software.

The same approach can be extended to support currency conversion and historical exchange rate features, giving you a versatile and scalable setup. As you continue building on this foundation, you’ll be well-equipped to develop more interactive and finance-focused applications.
If you need help or have any questions, please don't hesitate to reach out to us via our live chat or email support.

Please refer to the original tutorial published on our website: A Kotlin Tutorial to Fetch Market Data

Access The Latest Gold And Silver Prices With The Precious Metals API

Shridhar G Vatharkar — Wed, 04 Jun 2025 12:20:14 +0000

In today's ever-changing financial landscape, understanding the current prices of precious metals is essential for everyone, from investors to software developers and hobbyists alike. This guide will teach you how to retrieve gold and silver prices using the precious metals API.

This tutorial provides you with the resources to obtain accurate, up-to-the-minute information, regardless of your level of experience. It thoroughly explains how to get spot prices for precious metals data through TraderMade's REST API using various programming languages.

Why Precious Metals Data Matters

For hundreds of years, gold, silver, and other precious metals have been highly valued because they are rare and possess inherent worth. Precise pricing information for these metals is essential for making decisions in investment, manufacturing, jewelry, and financial analysis. Knowing how to retrieve and interpret data from the Precious Metals Prices API is necessary for informed decisions in these fields.

Methods to Receive Real-Time Financial Data

APIs

What is an API? An Application Programming Interface (API) enables you to obtain current financial data from various platforms programmatically.

Sign Up: To obtain an API key, you must register on the provider's website.
Choose a Plan: Pick a free or paid subscription that fits your data requirements.
Access Data: Then, use your API key to request the data with your chosen programming language, such as Python or JavaScript.

Python

This Python code uses the requests library to fetch real-time Gold prices from an API. The same request also retrieves Silver and Platinum prices, as demonstrated.

import requests
from pprint import PrettyPrinter

url = "https://marketdata.tradermade.com/api/v1/live"
currency = "XAUUSD,XAGUSD,XPTUSD,XPDUSD"
api_key = "API_KEY"  # Replace with your actual API key
querystring = {"currency": currency, "api_key": api_key}

response = requests.get(url, params=querystring)

# Check if the request was successful
if response.status_code == 200:
    pp = PrettyPrinter()
    pp.pprint(response.json())
else:
    print(f"Error {response.status_code}: {response.text}")

Java Programming

This Java program uses an HTTP GET request to get the Gold price data from the API. You can also expand it to fetch silver and other live precious metal prices. The "FetchMarketData" class sets up the API's URL and necessary parameters, like currency codes and the API key.

import org.apache.http.HttpEntity;
import org.apache.http.HttpHeaders;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
import java.io.IOException;

import org.json.JSONArray;
import org.json.JSONObject;

public class RESTClient {

    public static void main(String[] args) throws IOException {

        CloseableHttpClient httpClient = HttpClients.createDefault();

        try {

            HttpGet request = new HttpGet("https://marketdata.tradermade.com/api/v1/live
currency=XAUUSD&api_key=YOUR_API_KEY");
            CloseableHttpResponse response = httpClient.execute(request);
            try {
                HttpEntity entity = response.getEntity();
                if (entity != null) {
                    // return it as a String
                    String result = EntityUtils.toString(entity);
                    System.out.println(result);
                    JSONObject obj = new JSONObject(result);
                    JSONArray quotes = obj.getJSONArray("quotes");
                    System.out.println(quotes.toString());

                    for (int i = 0; i < quotes.length(); i++) {
                        JSONObject quote = quotes.getJSONObject(i);
                        System.out.println(" Quote " + quote.getString("base_currency") +
quote.getString("quote_currency") + " " + quote.getFloat("bid") + " " + quote.getFloat("ask"));
                    }

                }

            } finally {
                response.close();
            }
        } finally {
            httpClient.close();
        }
    }
}

Node.js Programming

This Node.js script uses the Axios library to get real-time Gold price data from the live API endpoint.

const axios = require('axios');
const util = require('util');

const url = "https://marketdata.tradermade.com/api/v1/live";
const currency = "XAUUSD";
const apiKey = "API_KEY";  // Replace with your actual API key

async function fetchMarketData() {

    try {
        const response = await axios.get(url, {
            params: {
                currency: currency,
                api_key: apiKey
            }
        });

        if (response.status === 200) {
                console.log(util.inspect(response.data, { depth: null, colors: true }));
            } else {
                  console.log("Error");
              }

          } catch (error) {
              console.error("Error");
          }

      }

      fetchMarketData();

C++ Programming

This C++ code snippet demonstrates how to get Gold prices from the live API endpoint using the libcurl library. The code starts by including the essential libcurl functions and standard input/output headers.

#include <curl/curl.h>
#include <stdio.h>

int main(void)
{
    CURL *curl;
    CURLcode res;

    // URL to be requested

    const char *url = "https://marketdata.tradermade.com/api/v1/live?currency=XAUUSD&api_key=API_KEY";

    // Initialize global curl environment
    curl_global_init(CURL_GLOBAL_DEFAULT);

    // Initialize curl session
    curl = curl_easy_init();

    if(curl) {

        // Set URL
        curl_easy_setopt(curl, CURLOPT_URL, url);

        // Perform the request
        res = curl_easy_perform(curl);

        // Check for errors
        if(res != CURLE_OK) {

            fprintf(stderr, "Request failed: %s
", curl_easy_strerror(res));
        } else {
            printf("Request was successful!
");
        }


        // Cleanup curl session
        curl_easy_cleanup(curl);
    }

    // Cleanup curl global environment
    curl_global_cleanup();

    return 0;
}

Conclusion

This tutorial explains how to access and use precious metals data from a JSON API using various programming languages. By using these methods, you can easily add live market data for Gold, Silver, Platinum, and Palladium prices into your applications and workflows.

For more examples in other programming languages and information on live and historical rates for world currencies, check our documentation page.

Also, follow our original tutorial published on our website: Get Live Gold and Silver Prices Instantly with Precious Metals API

You can explore more by reading the blog: How the Gold and Silver Prices Are Determined

Integrating Market Data into Your Go Applications with TraderMade's SDK

Shridhar G Vatharkar — Thu, 22 May 2025 16:27:49 +0000

Learn how to fetch real-time and historical market data in Go using TraderMade's easy-to-use SDK.

If you're a Go developer aiming to integrate real-time or historical market data into your applications, TraderMade's Go SDK offers a streamlined solution. Whether you're building trading tools, analytics dashboards, or financial applications, this SDK simplifies access to Forex, cryptocurrency, precious metals, and CFD data.

Getting Started

First, use Go version 1.21 or higher to support modules. Then, retrieve the TraderMade Go SDK:

go get github.com/tradermade/Go-SDK

After obtaining the SDK, initialize the REST client with your API key:

import (
    tradermade "github.com/tradermade/Go-SDK/rest"
)

client := tradermade.NewRESTClient("YOUR_API_KEY")

You can acquire your API key by signing up on the TraderMade website.

Fetching Live Forex Rates

To retrieve live rates for specific currency pairs, such as EUR/USD and Gold (XAU/USD):

currencyPairs := []string{"EURUSD", "XAUUSD"}
liveRates, err := client.GetLiveRates(currencyPairs)
if err != nil {
    log.Fatalf("Error fetching live rates: %v", err)
}

for _, quote := range liveRates.Quotes {
    fmt.Printf("Base: %s, Quote: %s, Bid: %f, Ask: %f, Mid: %f\n",
        quote.BaseCurrency, quote.QuoteCurrency, quote.Bid, quote.Ask, quote.Mid)
}

This code initializes the REST client, defines the desired currency pairs, fetches their live rates, and prints out the base currency, quote currency, bid, ask, and mid prices.

Performing Currency Conversion

To convert an amount from one currency to another in real-time:

convertResult, err := client.ConvertCurrency("EUR", "GBP", 1000.0)
if err != nil {
    log.Fatalf("Error fetching conversion data: %v", err)
}

fmt.Printf("Converted %s to %s:\n", convertResult.BaseCurrency, convertResult.QuoteCurrency)
fmt.Printf("Quote: %f\n", convertResult.Quote)
fmt.Printf("Total: %f\n", convertResult.Total)
fmt.Printf("Requested Time: %s\n", convertResult.RequestedTime)
fmt.Printf("Timestamp: %d\n", convertResult.Timestamp)

This snippet converts 1,000 EUR to GBP, displaying the conversion rate, total amount, request time, and timestamp.

Retrieving Hourly Time Series Data

For historical analysis or charting, you might need time series data. Here's how to fetch hourly data for EUR/USD:

timeSeriesDataHourly, err := client.GetTimeSeriesData("EURUSD", "2024-10-01 10:00", "2024-10-02 11:00", "hourly", 4)
if err != nil {
    log.Fatalf("Error fetching hourly time series data: %v", err)
}

for _, data := range timeSeriesDataHourly.TimeSeries {
    fmt.Printf("Time: %s, Open: %f, High: %f, Low: %f, Close: %f\n",
        data.Date, data.Open, data.High, data.Low, data.Close)
}

This code fetches hourly data between October 1, 2024, 10:00 AM and October 2, 2024, 11:00 AM, printing out the open, high, low, and close prices for each hour.

Example: Real-Time Forex Data with WebSocket

WebSocket feeds are ideal for ultra-low-latency applications like trading terminals or automated bots. TraderMade supports WebSocket streaming for live updates.
Here’s a quick example:

package main

import (
    "github.com/tradermade/Go-SDK/websocket"
)

func main() {
    ws := websocket.NewWebSocketClient("YOUR_API_KEY")
    ws.Connect()
    ws.Subscribe([]string{"EURUSD", "GBPUSD"})

    for msg := range ws.MessageChannel {
        fmt.Printf("Received tick: %v\n", msg)
    }
}

This function opens a WebSocket connection and allows you to subscribe to live updates for EUR/USD and GBP/USD. Each incoming tick is pushed to a channel for real-time processing.

Using the WebSocket Client

Initialize the WebSocket client with your API key and the currency pairs you wish to subscribe to.
Configure connection settings, including automatic retries and reconnection behavior.
Set up event handlers:

Connection Handler: Triggered when the WebSocket successfully connects.

client.SetConnectedHandler(func(connectedMsg tradermadews.ConnectedMessage) {
    fmt.Printf("WebSocket connected: %s\n", connectedMsg.Message)
})

Quote Message Handler: Processes incoming real-time market data such as bid/ask quotes.

client.SetMessageHandler(func(quote tradermadews.QuoteMessage, humanTimestamp string) {
    fmt.Printf("Received quote: Symbol=%s Bid=%.5f Ask=%.5f Timestamp=%s (Human-readable: %s)\n",
        quote.Symbol, quote.Bid, quote.Ask, quote.Ts, humanTimestamp)
})

Reconnection Handler: Notifies you when the client is attempting to reconnect.

client.SetReconnectionHandler(func(attempt int) {
    fmt.Printf("Reconnecting... (Attempt %d)\n", attempt)
})

Enable graceful shutdown on user interrupt (e.g., Ctrl+C).

c := make(chan os.Signal, 1)
signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
<-c
fmt.Println("Shutting down WebSocket client...")

Summary

Connects to the WebSocket feed using secure credentials.
Receives real-time updates for Forex, stock indices, and precious metals.
Automatically handles disconnections and attempts to reconnect.
Provides handlers for key lifecycle events: connection, message, and reconnection.
Supports clean shutdowns for a robust and reliable client experience.

The WebSocket client is ideal for apps needing real-time FX market data with minimal latency.

Additional Resources

TraderMade's Go SDK also supports fetching historical data, performing currency conversions, and retrieving time series data. For comprehensive details and advanced usage, refer to the TraderMade REST API Documentation and explore their GitHub repository.

By integrating TraderMade's Go SDK into your projects, you can efficiently access a wide range of financial data, enhancing the functionality and reliability of your applications.

Wrapping Up

TraderMade’s Go SDK equips you with everything needed to tap into a world of financial data with minimal effort. From real-time streaming to historical insights and currency conversion, it's easily accessible through a clean, idiomatic Go interface.

Please refer to the originally published tutorial on our website: Accessing Market Data with TraderMade's Go Library

Also, go through the other Golang Tutorials:
Your First Golang REST API Client

Your First Golang WebSocket Client

Using Python & WebSockets to Generate Live OHLC Forex Data

Shridhar G Vatharkar — Wed, 30 Apr 2025 12:03:24 +0000

This tutorial guides you through building a real-time Forex OHLC data generator using Python and WebSockets. Learn how to stream FX prices, convert them to OHLC format, and save the results with clean, efficient code.

Accessing real-time market data is essential for modern trading and financial analytics. WebSockets provide an efficient method for live, continuous data streaming. In this tutorial, we’ll walk through how to use Python to connect to a WebSocket stream, collect Forex pricing data, and convert it into OHLC (Open, High, Low, Close) bars.

What You'll Need

Before we start, ensure you have:

Python installed on your machine.
Basic understanding of how to use Python.

Get Your API Key

To stream live Forex data, you’ll need a TraderMade API key. Sign up for a 14-day free trial and find your API key in your account dashboard. Refer to the platform's documentation for helpful examples and guidance.

Let’s Get Started

We’ll be using the tradermade Python package to access live FX rates. The data will be processed into OHLC format and written to a text file in real-time.

Step 1: Install the TraderMade Package

Install the required package using pip:

pip install tradermade

Step 2: Import Required Libraries

Begin by importing the modules needed to handle WebSocket streaming, manipulate data structures, and format time.

from tradermade import stream
import json
from datetime import datetime
from collections import defaultdict

Step 3: Initialize Data Structures

Prepare dictionaries and variables for managing OHLC values, tracking intervals, and defining output behaviour.

ohlc_data = defaultdict(lambda: defaultdict(lambda: {'open': 0, 'high': 0, 'low': 0, 'close': 0}))
previous_interval = None
count = 0
interval = 1
format = '%Y-%m-%d %H:%M'
output_file = 'ohlc_output.txt'

Step 4: Define Utility Functions

Now we’ll create three utility functions to handle time rounding, file writing, and updating OHLC values. Each function plays a specific role:

round_timestamp(timestamp, interval_minutes)

This function rounds any incoming timestamp to the nearest defined time interval.

Useful for aligning price data into fixed OHLC intervals.
Replaces seconds and microseconds, rounding down to the nearest minute block.

save_to_file(data)

Appends a string to the output text file.

Ensures processed OHLC entries are saved for future use.
Opens the file in append mode and writes the data with a space separator.

update_ohlc_data(current_interval, currency, mid)

Updates the OHLC values for each currency pair during the given interval.

If no data exists yet for the interval and currency, it initializes values with the current price.
The 'open' is set at the start, and the 'high'/'low' are tracked as data flows in.
The 'close' is updated every time new data is received in that interval.

def round_timestamp(timestamp, interval_minutes):
    rounded_minutes = (timestamp.minute // interval_minutes) * interval_minutes
    rounded_time = timestamp.replace(second=0, microsecond=0, minute=rounded_minutes)
    return rounded_time
def save_to_file(data):
    with open(output_file, 'a') as f:
        f.write(data + ' ')
def update_ohlc_data(current_interval, currency, mid):
    if current_interval not in ohlc_data:
        ohlc_data[current_interval] = defaultdict(lambda: {'open': mid, 'high': mid, 'low': mid, 'close': mid})
    elif currency not in ohlc_data[current_interval]:
        ohlc_data[current_interval][currency] = {'open': mid, 'high': mid, 'low': mid, 'close': mid}
    else:
        ohlc_data[current_interval][currency]['high'] = max(ohlc_data[current_interval][currency]['high'], mid)
        ohlc_data[current_interval][currency]['low'] = min(ohlc_data[current_interval][currency]['low'], mid)
        ohlc_data[current_interval][currency]['close'] = mid

Step 5: Connect to the WebSocket Stream

You’re now ready to initiate the WebSocket connection. Use your API key and specify the currency symbols you'd like to stream.

stream.set_ws_key("API_KEY")  # Replace with your actual API key
stream.set_symbols("USDJPY,EURGBP")
stream.stream_data(process_data)
stream.connect()

Step 6: Run Your Script

Finally, execute the script to start processing live data and generating OHLC output.

python websocket_ohlc.py

After running the program, a file named ohlc_output.txt will appear in your working directory. It will contain a growing log of OHLC data points based on your specified interval.

Conclusion

You’ve just built a working solution for streaming live Forex data and transforming it into OHLC bars using Python and WebSockets. This setup can serve as a foundation for real-time dashboards, trading bots, or analytical tools.

Please refer to:
The tutorial originally published on our website: Create Real-time OHLC Data with Python WebSocket

Also, watch the video tutorial: Real-time FX Ticks to OHLC Bars With Python.

Converting Real-Time Forex Data to OHLC Bars Using PHP

Shridhar G Vatharkar — Tue, 25 Mar 2025 11:01:09 +0000

Ever wanted to handle real-time data using WebSockets? WebSockets allow seamless live data streaming, making them perfect for applications like Forex trading.

This guide'll capture real-time Forex data using a WebSocket connection and transform it into minute-by-minute Open, High, Low, and Close (OHLC) bars. These bars provide valuable insights for traders and analysts tracking short-term market trends.

Even if you're new to PHP, don’t worry! We'll take it step by step.
Please take a look at WebSockets Documentation for details on usage and integration.

Let’s dive in.

Setting Up Your Environment

Before we get into the coding, let’s ensure our setup is ready.
Check PHP Installation: Open a terminal and run:

php -version

If PHP isn’t installed, download and install it from the official PHP website.

Get a WebSocket API Key: Sign up for a free 14-day WebSocket trial via your TraderMade dashboard.
Install Composer (PHP Dependency Manager): Navigate to your project folder and install the WebSocket library:

composer require textalk/websocket

Creating the WebSocket Client in PHP

Step 1: Include Dependencies

First, include Composer's autoload file to manage dependencies efficiently:

require_once("vendor/autoload.php");

Step 2: Establish WebSocket Connection

We need to connect to the WebSocket server. This function handles the connection:

function connectWebSocket() {
    try {
        $client = new WebSocketClient("wss://marketdata.tradermade.com/feedadv");

        $message = $client->receive();
        echo $message;

        $client->text("{\"userKey\":\"api_key\", \"symbol\":\"GBPUSD,EURUSD\"}");

        return $client;
    } catch (WebSocketTimeoutException $e) {
        echo "WebSocket timeout. Reconnecting...\n";
        sleep(5);
        return connectWebSocket();
    }
}

This function:

Connects to the WebSocket server.
Requests real-time data for specific currency pairs.
Implements auto-reconnection in case of timeouts.

Step 3: Receive and Process WebSocket Data

Once connected, we listen for price updates and store them:

$client = connectWebSocket();
$dataArray = [];

while (true) {
    try {
        $message = $client->receive();

        if ($message !== "connected") {
            $decoded_json = json_decode($message, true);

            $bid = $decoded_json['bid'] ?? null;
            $ask = $decoded_json['ask'] ?? null;

            if ($bid !== null && $ask !== null) {
                $mid = ($bid + $ask) / 2;

                echo "{$decoded_json['symbol']} {$decoded_json['ts']} $bid $ask $mid\n";

                $dataArray[] = [
                    'symbol' => $decoded_json['symbol'],
                    'ts' => $decoded_json['ts'],
                    'bid' => $bid,
                    'ask' => $ask,
                    'mid' => $mid
                ];

                file_put_contents('websocket_data.json', json_encode($dataArray, JSON_PRETTY_PRINT));
            }
        }
    } catch (WebSocketTimeoutException $e) {
        echo "Timeout. Reconnecting...\n";
        sleep(5);
        $client = connectWebSocket();
    } catch (Exception $e) {
        echo "Error: " . $e->getMessage();
    }
}

This script:

Reads live Forex data.
Extracts bid and ask prices.
Saves data into websocket_data.json.
Handles errors and automatically reconnects if necessary.

Step 4: Load Data from JSON File

Before processing, we ensure our file has valid data:

$jsonFilePath = 'websocket_data.json';
$jsonData = file_get_contents($jsonFilePath);
$forexData = json_decode($jsonData, true);

if (empty($forexData)) {
    echo json_encode(["error" => "No data found in $jsonFilePath"]);
    exit;
}

Step 5: Convert Raw Data to OHLC Bars

Now, let’s transform the tick data into minute-based OHLC bars:

$interval = 60;
$ohlcData = [];

foreach ($forexData as $data) {
    $timestamp = strtotime($data['ts']);
    $roundedTimestamp = floor(($timestamp + (5 * 3600) + (30 * 60)) / $interval) * $interval;

    if (!isset($ohlcData[$roundedTimestamp])) {
        $ohlcData[$roundedTimestamp] = [
            'timestamp' => $roundedTimestamp + (5 * 3600) + (30 * 60),
            'low' => $data['bid'],
            'high' => $data['bid'],
            'open' => $data['bid'],
            'close' => $data['bid']
        ];
    } else {
        $ohlcData[$roundedTimestamp]['low'] = min($ohlcData[$roundedTimestamp]['low'], $data['bid']);
        $ohlcData[$roundedTimestamp]['high'] = max($ohlcData[$roundedTimestamp]['high'], $data['bid']);
        $ohlcData[$roundedTimestamp]['close'] = $data['bid'];
    }
}

This:

Rounds timestamps to the nearest minute.
Generates OHLC bars for each minute.

Step 6: Save OHLC Data to JSON

Finally, we store the OHLC bars:

$jsonFilePathOutput = 'ohlc_data.json';

if (!file_put_contents($jsonFilePathOutput, json_encode(array_values($ohlcData), JSON_PRETTY_PRINT))) {
    echo json_encode(["error" => "Failed to write OHLC data"]);
    exit;
}

echo json_encode(["success" => "OHLC data saved to $jsonFilePathOutput"]);

Running the Script

To execute, run:

php webSocket_client.php

Your script will now fetch live Forex data and convert it into OHLC bars. 🚀

Sample Outputs

Raw Forex Data:

[
    {"symbol": "GBPUSD", "ts": "1712130980714", "bid": 1.25664, "ask": 1.25668},
    {"symbol": "EURUSD", "ts": "1712130982178", "bid": 1.07712, "ask": 1.07712}
]

OHLC Data Output:

[
    {"timestamp": "1970-01-01 05:30:00", "open": 1.25664, "high": 1.25665, "low": 1.25664, "close": 1.25665}
]

Conclusion
Congratulations! You’ve successfully streamed real-time Forex data and transformed it into OHLC bars. Try tweaking the script and adding new features. Happy coding! 🎉

Effortlessly Calculate 100+ Technical Indicators in Python – A Quick Guide

Shridhar G Vatharkar — Mon, 03 Mar 2025 14:29:01 +0000

Technical indicators are essential for detailed market analysis and data-driven decision-making. This tutorial will walk you through retrieving historical forex data using the TraderMade API and computing key technical indicators with the Python TA-Lib library. By the end, you will have the necessary tools to integrate these indicators into your applications, platforms, and trading systems.

Our API provides real-time data on a wide range of tradable assets, including over 32 US stocks, 10 significant indices, precious metals, and access to more than 100 currencies (explore currencies & CFDs).

Before proceeding, it is essential to install the TA-Lib library. Since pip often fails to build this library on specific systems, mainly Windows, we will demonstrate an alternative method using a wheel file.

Linux users should refer to the official TA-Lib library. Windows users should visit the TA-Lib Windows wheel page, download the appropriate wheel file based on their Python version and system architecture (32-bit or 64-bit), and ensure that pip is executed from the exact location as the downloaded file.

In the example below, we have downloaded a wheel file compatible with 64-bit computers running Python 3.10. Thus, the following download method is recommended:

pip install requests pandas 
pip install TA_Lib-0.4.32-cp310-cp310-win_amd64.whl

After installing the library, register with TraderMade to get your free Forex API key. Next, install TraderMade for seamless integration.

pip install tradermade

Detailed Guide

Step 1: Import Required Libraries

Start by importing the essential data retrieval, processing, and analysis libraries.

import talib
import tradermade as tm

Step 2: Configure Your API Key

Insert your TraderMade API key to enable authentication.

# Replace Your TraderMade API key
tm.set_rest_api_key("api_key")

Step 3: Define a Function to Retrieve Market Data

Develop a function to fetch historical forex data for a selected currency pair over a specified date range.

tm.timeseries(currency='EURUSD', start="2024-08-05",end="2024-11-04",interval="daily",fields=["open", "high", "low","close"])

Step 4: Compute Essential Technical Indicators

TA-Lib offers numerous indicators, including candlestick pattern recognition. Below are some key technical indicators widely used by traders:

Simple Moving Average (SMA)
Smooths price data over a defined period (e.g., 20 days) to identify trends.
Relative Strength Index (RSI)
A momentum oscillator that measures the speed and change of price movements, helping to detect overbought or oversold conditions.
Moving Average Convergence Divergence (MACD)
Analyzes the relationship between two moving averages of a financial instrument, aiding in spotting potential market momentum shifts and generating buy/sell signals.

# Calculate technical indicators using TA-Lib
df['SMA_20'] = talib.SMA(df['close'], timeperiod=20) # 20-period Simple Moving Average

df['RSI_14'] = talib.RSI(df['close'], timeperiod=14) # 14-period Relative Strength Index
df['MACD'], df['MACD_signal'], df['MACD_hist'] = talib.MACD(df['close'],
                                                                  fastperiod=12,
                                                                  slowperiod=26,
                                                                  signalperiod=9)

Step 5: View the Results

Finally, the last few rows of the DataFrame will be displayed to examine the OHLC data alongside the computed indicators.

df = df.set_index("date")
df = df.dropna()
print(df.head())

              open     high      low    close    SMA_20     RSI_14      MACD  MACD_signal  MACD_hist

date
2024-09-19  1.11172  1.11790  1.10686  1.11621  1.109625  62.318384  0.003357     0.003439  -0.000083
2024-09-20  1.11626  1.11820  1.11362  1.11631  1.109472  62.410940  0.003576     0.003467   0.000109
2024-09-23  1.11646  1.11674  1.10834  1.11114  1.109223  54.902623  0.003295     0.003432  -0.000138
2024-09-24  1.11114  1.11811  1.11036  1.11797  1.109199  61.493338  0.003582     0.003462   0.000119
2024-09-25  1.11799  1.12141  1.11217  1.11329  1.109263  55.507279  0.003392     0.003448  -0.000056

Complete Code

Below is the full implementation—review it carefully.

# Display the data with indicators

import talib
import tradermade as tm

tm.set_rest_api_key("api_key")

df = tm.timeseries(currency='EURUSD', start="2024-08-05",end="2024-11-04",interval="daily",fields=["open", "high", "low","close"])

df['SMA_20'] = talib.SMA(df['close'], timeperiod=20) # 20-period Simple Moving Average

df['RSI_14'] = talib.RSI(df['close'], timeperiod=14) # 14-period Relative Strength Index
df['MACD'], df['MACD_signal'], df['MACD_hist'] = talib.MACD(df['close'],
                                                                  fastperiod=12,
                                                                  slowperiod=26,
                                                                  signalperiod=9)

df = df.set_index("date")
df = df.dropna()
print(df.head())

The Bottom Line

This tutorial demonstrated how to retrieve historical forex data using the TraderMade SDK and compute key technical indicators such as SMA, RSI, and MACD with the TA-Lib library. However, this is just the beginning—TA-Lib offers over 100 indicators, including candlestick patterns, applicable to various markets.

If you found this guide helpful, share our tutorials or reference them in your blogs. Don't hesitate to reach out for any questions or assistance with TA-Lib installation. Happy coding!

Also, please review the tutorial initially published on our website: 100+ Technical Indicators in Python.
Here is a tutorial video: Technical Indicators in Python Video Tutorial.

The Future of Forex Trading: AI and Real-Time Data Integration

Shridhar G Vatharkar — Fri, 21 Feb 2025 10:05:48 +0000

Discover how AI & real-time data are revolutionizing forex trading. Learn the benefits of Forex APIs for better trading performance.

In the dynamic world of forex trading, staying ahead requires more than traditional strategies. Integrating Artificial Intelligence (AI) and real-time data has emerged as a game-changer, offering traders unprecedented insights and execution capabilities.

Central to this evolution is using Forex APIs from reputable data vendors, which significantly enhance the performance of real-time trading applications.

The Rise of AI in Forex Trading

AI has revolutionized many industries, including forex trading. It analyzes massive datasets to detect patterns and trends that human traders might overlook. This ability enables:

Predictive Analysis: AI models forecast currency movements based on historical data and market indicators, enabling traders to make informed decisions.
Automated Trading: AI-driven systems can execute trades autonomously, responding to millisecond market changes, which is crucial in the fast-paced forex environment.
Risk Management: AI assesses potential risks by evaluating market volatility and other factors, helping traders mitigate potential losses.

The Importance of Real-Time Data

In forex trading, timing is everything. Access to real-time data ensures that traders have the most current information, allowing them to:

React Swiftly: Immediate data access enables prompt responses to market shifts, capitalizing on opportunities.
Enhance Accuracy: Real-time forex data reduces the lag between market events and trader reactions, leading to more precise trading decisions.
Improve Strategy Testing: Traders can backtest strategies against live data, refining their approaches for better outcomes.

Leveraging Forex APIs for Enhanced Performance

Forex Application Programming Interfaces (APIs) bridge trading platforms and data sources. By integrating Forex APIs from reputable data vendors, developers and analysts can:

Access Comprehensive Data: Reliable APIs provide extensive historical and real-time data, essential for thorough market analysis.
Ensure Data Integrity: Partnering with reputable vendors guarantees accurate and consistent data, which is critical for practical trading strategies.
Customize Trading Solutions: APIs allow for developing tailored trading applications that align with specific trading styles and requirements.

Choosing the Right Data Vendor

Selecting a reputable data vendor is paramount. Consider the following factors:

Data Accuracy: Ensure the vendor offers precise and up-to-date information.
Reliability: The vendor should have a robust infrastructure to access uninterrupted data.
Support Services: Quality customer support is essential for addressing any issues promptly.

Conclusion

The fusion of AI and real-time data integration reshapes the forex trading landscape. By harnessing the power of AI and utilizing Forex APIs from reputable data vendors, traders can enhance their decision-making processes, execute timely trades, and develop sophisticated, real-time trading applications. Embracing these technological advancements is not just an option but a necessity for those aiming to excel in the modern forex market.

Building Your First C++ WebSocket Client

Shridhar G Vatharkar — Tue, 18 Feb 2025 14:27:28 +0000

Welcome to our step-by-step tutorial on accessing real-time Forex, CFD, and Crypto data using TraderMade’s WebSocket with C++. In the fast-paced financial world, staying updated with live data is essential for making informed decisions. This guide will walk you through the entire process seamlessly.

Getting Started

First, sign up for a TraderMade account and get a free API key for a 2-week trial. This key is your gateway to real-time market data. Once you have it, you’ll use it to authenticate your requests. Reviewing the Streaming Forex API documentation will also help you understand the available data and how to use it effectively.

Setting Up Your Coding Environment

C++ development is more convenient in a Linux environment. You can install the Ubuntu app from the search bar if you're using Windows.

For those new to C++, installing the GCC compiler is essential for running C++ programs. We’ll break everything down into simple steps to make the setup process smooth and efficient.

Let’s dive in and start building!

sudo apt update

If you haven’t set a password or can't remember it, you can reset it using the following command:

sudo passwd

With our system up to date, let's go ahead and install the essential tools for compilation by running the following command:

sudo apt install build-essential

To check the version of GCC you're working with, run the following command:

gcc --version

Now that the C++ compiler is set up, let's import the necessary libraries to run our program.

git clone https://github.com/zaphoyd/websocketpp.git
cd websocketpp

This will fetch the WebSocket++ library directly from its GitHub repository. Additionally, we’ll need to install two more libraries: Boost and SSL. You can do so by running the following commands:

sudo apt-get install libboost-all-dev
sudo apt-get install libssl-dev

Writing Code

Now, let's include the libraries necessary to run our program.

#include <websocketpp/config/asio_client.hpp> 
#include <websocketpp/client.hpp> 
#include <iostream>

Next, we'll define aliases using typedef for easier reference in our code. The websocketpp is a namespace within the WebSocket++ library. The first line represents a complex type that enables WebSocket client support with Asio (for asynchronous I/O) and TLS (for secure communication).

In the second line, shared_ptr is a smart pointer from the WebSocket++ library that automatically manages memory for the object. The type of object it will manage is specified after the < symbol. context_ptr is an alias that simplifies using the longer type in the second line, making the code more readable and concise.

typedef websocketpp::client<websocketpp::config::asio_tls_client> client;
typedef websocketpp::lib::shared_ptr<websocketpp::lib::asio::ssl::context> context_ptr;

The code below should now be clearer. In the first two lines, _1 and _2 represent the first and second arguments, which are passed to the WebSocket++ function objects and callbacks. The third line defines the bind function, which allows you to create function objects using placeholders.

using websocketpp::lib::placeholders::_1;
using websocketpp::lib::placeholders::_2;
using websocketpp::lib::bind;

Now that we've defined the placeholders and bound them to callback functions for handling WebSocket events, we'll explain the functions for each event that we might encounter after establishing a connection.

First, we define the on_open callback function, which is triggered when a WebSocket connection is successfully made. It takes two parameters: a handle to the connection (hdl) and a pointer to the WebSocket client (client* c).
The second line of the code is a simple print statement. Following that, we include an error code for error handling and a connection pointer (connection_ptr). The if statement checks for any errors when retrieving the connection pointer. We send a string payload as text over the WebSocket connection if no error is found.

void on_open(websocketpp::connection_hdl hdl, client* c) {
    std::cout << "WebSocket connection opened!" << std::endl;

    websocketpp::lib::error_code ec;
    client::connection_ptr con = c->get_con_from_hdl(hdl, ec);
    if (ec) {
        std::cout << "Failed to get connection pointer: " << ec.message() << std::endl;
        return;
    }
    std::string payload = "{"userKey":"API_KEY", "symbol":"EURUSD,GBPUSD"}";
    c->send(con, payload, websocketpp::frame::opcode::text);
}

Now that we're familiar with the on_open callback function, the next part of the code is easy to grasp. The on_message callback function also takes two parameters; the second is message_ptr msg, which contains the Forex data received. We then print this data inside the on_message function.

The on_fail and on_close callbacks are straightforward and don't require much explanation to proceed to the next section of the code.

void on_message(websocketpp::connection_hdl, client::message_ptr msg) {
    std::cout << "Received message: " << msg->get_payload() << std::endl;

}
void on_fail(websocketpp::connection_hdl hdl) {
    std::cout << "WebSocket connection failed!" << std::endl;
}
void on_close(websocketpp::connection_hdl hdl) {
    std::cout << "WebSocket connection closed!" << std::endl;
}

The next function configures the SSL context for secure communication in the WebSocket client. We won’t delve into the specifics of this function; instead, we’ll jump straight into the main function, where everything integrates.

context_ptr on_tls_init(const char * hostname, websocketpp::connection_hdl) {
    context_ptr ctx = websocketpp::lib::make_shared<boost::asio::ssl::context>(boost::asio::ssl::context::sslv23);

    try {
        ctx->set_options(boost::asio::ssl::context::default_workarounds |
                         boost::asio::ssl::context::no_sslv2 |
                         boost::asio::ssl::context::no_sslv3 |
                         boost::asio::ssl::context::single_dh_use);

        ctx->set_verify_mode(boost::asio::ssl::verify_none);
    } catch (std::exception& e) {
        std::cout << "TLS Initialization Error: " << e.what() << std::endl;
    }
    return ctx;
}

In the main function, we define the client, then specify the URL to connect to the Forex WebSocket and retrieve sub-second data.

int main(int argc, char* argv[]) {
    client c;

    std::string hostname = "marketdata.tradermade.com/feedadv";
    std::string uri = "wss://" + hostname;
}

We set up a try block inside the main function to handle possible exceptions.

int main(int argc, char* argv[]) {
....
try {

    } catch (websocketpp::exception const & e) {
        std::cout << "WebSocket Exception: " << e.what() << std::endl;
    }

}

Within the try block, we begin by configuring the WebSocket++ client and initializing Asio for asynchronous operations. Next, we set up message and event handlers, including the TLS (Transport Layer Security) initialization handler.

We then declare an error object and check for issues while retrieving the connection. Lastly, we create a connection using c.connect(con) and initiate the WebSocket event loop with c.run(), which handles the asynchronous events.

try {
      // Configure WebSocket++ client
      c.set_access_channels(websocketpp::log::alevel::all);
      c.clear_access_channels(websocketpp::log::alevel::frame_payload);
      c.set_error_channels(websocketpp::log::elevel::all);

      c.init_asio();

      // Set message, TLS initialization, open, fail, and close handlers
      c.set_message_handler(&on_message);

      c.set_tls_init_handler(bind(&on_tls_init, hostname.c_str(), ::_1));

      c.set_open_handler(bind(&on_open, ::_1, &c));
      c.set_fail_handler(bind(&on_fail, ::_1));


      c.set_close_handler(bind(&on_close, ::_1));
      // Enable detailed error logging

      c.set_error_channels(websocketpp::log::elevel::all);          

      websocketpp::lib::error_code ec;

      client::connection_ptr con = c.get_connection(uri, ec);

      if (ec) {
                 std::cout << "Could not create connection because: " << ec.message() << std::endl;
                 return 0;
              }

      // Create a connection to the specified url
      c.connect(con);

      c.run();

    } catch (websocketpp::exception const & e) {
      std::cout << "WebSocket Exception: " << e.what() << std::endl;
    }

After writing the code, save it in a file named FX_WebSocket.cpp (or any name you prefer). Then, compile the program using the following command:

g++ -o ./streaming_forex  FX_WebSocket.cpp -lssl -lcrypto -pthread

Your program should now be compiled into the streaming_forex file. To run the file, use the following command:

./streaming_forex

Voila! You should now see a WebSocket connection with your subscribed Forex data. Remember to replace your API key when sending the payload using the on_open function.

Output

Great! Our output is ready.

Received message: Connected
Received message: {"symbol":"GBPUSD","ts":"1708403061263","bid":1.25867,"ask":1.25871,"mid" :1.25869}
Received message: {"symbol":"GBPUSD","ts":"1708403070164","bid":1.25867,"ask":1.2587,"mid":1.258685}
Received message: {"symbol":"GBPUSD","ts":"1708403070951","bid":1.25866,"ask":1.2587,"mid":1.25868}
Received message: {"symbol":"GBPUSD","ts":"1708403071022","bid":1.25866,"ask":1.25871,"mid":1.258685}
Received message: {"symbol":"GBPUSD","ts":"1708403071213","bid":1.25866,"ask":1.2587,"mid":1.25868}
Received message: {"symbol":"GBPUSD","ts":"1708403071400","bid":1.25866,"ask":1.25869,"mid":1.258675}
Received message: {"symbol":"EURUSD","ts":"1708403071512","bid":1.07687,"ask":1.07691,"mid":1.07689}
Received message: {"symbol":"EURUSD","ts":"1708403071524","bid":1.07687,"ask":1.0769,"mid":1.076885}
Received message: {"symbol":"GBPUSD","ts":"1708403072138","bid":1.25866,"ask":1.2587,"mid":1.25868}
Received message: {"symbol":"GBPUSD","ts":"1708403074548","bid":1.25866,"ask":1.25869,"mid":1.258675}
Received message: {"symbol":"GBPUSD","ts":"1708403074618","bid":1.25866,"ask":1.2587,"mid":1.25868}
Received message: {"symbol":"GBPUSD","ts":"1708403074921","bid":1.25866,"ask":1.25869,"mid":1.258675}
Received message: {"symbol":"GBPUSD","ts":"1708403075348","bid":1.25866,"ask":1.2587,"mid":1.25868}
Received message: {"symbol":"GBPUSD","ts":"1708403077083","bid":1.25866,"ask":1.25869,"mid":1.258675}

Congratulations! You've successfully built a real-time Forex data fetcher using C++ and WebSocket++. Feel free to explore and adjust the code to suit your needs, and enjoy access to live financial data at your fingertips.

Full Code

Below is the complete code:

#include <websocketpp/config/asio_client.hpp>
#include <websocketpp/client.hpp>
#include <iostream>

typedef websocketpp::client<websocketpp::config::asio_tls_client> client;
typedef websocketpp::lib::shared_ptr<websocketpp::lib::asio::ssl::context> context_ptr;

using websocketpp::lib::placeholders::_1;
using websocketpp::lib::placeholders::_2;
using websocketpp::lib::bind;

void on_open(websocketpp::connection_hdl hdl, client* c) {
    std::cout << "WebSocket connection opened!" << std::endl;

     websocketpp::lib::error_code ec;
    client::connection_ptr con = c->get_con_from_hdl(hdl, ec);

    if (ec) {
        std::cout << "Failed to get connection pointer: " << ec.message() << std::endl;
        return;
    }

  std::string payload = "{"userKey":"API_KEY", "symbol":"EURUSD,GBPUSD"}";
    c->send(con, payload, websocketpp::frame::opcode::text);
}

void on_message(websocketpp::connection_hdl, client::message_ptr msg) {
    std::cout << "Received message: " << msg->get_payload() << std::endl;
}

void on_fail(websocketpp::connection_hdl hdl) {
    std::cout << "WebSocket connection failed!" << std::endl;
}

void on_close(websocketpp::connection_hdl hdl) {
    std::cout << "WebSocket connection closed!" << std::endl;
}

context_ptr on_tls_init(const char * hostname, websocketpp::connection_hdl) {
    context_ptr ctx = websocketpp::lib::make_shared<boost::asio::ssl::context>(boost::asio::ssl::context::sslv23);

    try {
        ctx->set_options(boost::asio::ssl::context::default_workarounds |
                         boost::asio::ssl::context::no_sslv2 |
                         boost::asio::ssl::context::no_sslv3 |
                         boost::asio::ssl::context::single_dh_use);

    } catch (std::exception& e) {
        std::cout << "TLS Initialization Error: " << e.what() << std::endl;
    }


    return ctx;
}

int main(int argc, char* argv[]) {
    client c;

    std::string hostname = "marketdata.tradermade.com/feedadv";
    std::string uri = "wss://" + hostname;


    try {
        c.set_access_channels(websocketpp::log::alevel::all);
        c.clear_access_channels(websocketpp::log::alevel::frame_payload);
        c.set_error_channels(websocketpp::log::elevel::all);
        c.init_asio();


        c.set_message_handler(&on_message);
        c.set_tls_init_handler(bind(&on_tls_init, hostname.c_str(), ::_1));

        c.set_open_handler(bind(&on_open, ::_1, &c));
        c.set_fail_handler(bind(&on_fail, ::_1));
        c.set_close_handler(bind(&on_close, ::_1));
        c.set_error_channels(websocketpp::log::elevel::all);  // Enable detailed error logging

        websocketpp::lib::error_code ec;
        client::connection_ptr con = c.get_connection(uri, ec);
        if (ec) {
            std::cout << "Could not create connection because: " << ec.message() << std::endl;
            return 0;
        }

        c.connect(con);

        c.run();
    } catch (websocketpp::exception const & e) {
        std::cout << "WebSocket Exception: " << e.what() << std::endl;
    }

}

Next Steps

TraderMade’s Live Streaming API is an easy-to-use and powerful resource for building digital solutions and gaining valuable insights. You can use it in C++ to create impressive applications, and we’d love to see what you make!

Register and explore the details in the Live Streaming API documentation to get started. If you need any help, feel free to reach out via live chat or email us at support@tradermade.com. We're here to help you on your development journey.

Creating Minute-Based HLOC Bars from WebSockets Data with NodeJS

Shridhar G Vatharkar — Thu, 23 Jan 2025 12:15:20 +0000

Learn how to convert WebSocket data into minute-based HLOC bars using Node.js. Step-by-step guide for efficient financial data processing.

As WebSockets continues to gain traction in today's tech world, curious developers are eager to explore this exciting technology. Setting up a server-client setup and experimenting with WebSocket data can be pretty thrilling, but the real magic happens when you access WebSockets, which delivers real-time, actionable data.

This tutorial will focus on something super helpful: turning raw WebSocket data into minute-based High, Low, Open, and Close (HLOC) bars. This transformation helps developers and analysts track market trends in one-minute intervals.

The best part? You don't need to be a Node.js pro to follow along! While some essential programming experience (especially with JavaScript) might help, it's unnecessary. Are you not a JavaScript fan? No worries—you can also explore WebSocket implementations in Python or Golang.

Before starting the setup and coding, let's clarify the plan. We'll fetch real-time Forex data via a WebSocket connection and convert it into minute-based HLOC bars using the TraderMade WebSocket API. The documentation provides more details.

Here's our game plan, broken into four simple steps:

Install NodeJS and set up your environment.
Sign up for a free trial and grab your API key.
Build a Node.js WebSocket client to receive data.
Process WebSocket data into minute-based HLOC bars with Node.js.

Want a quick walkthrough? Check out this YouTube video.

Step 1: Install NodeJS and Set Up Your Environment

For Windows and Mac users: Download and install NodeJS.
For Linux fans: Use apt-get to install NodeJS.

Once NodeJS is installed, open your command prompt (Windows) or terminal (Mac/Linux). Now, let's get the necessary dependencies in place. For our client, we're using the "ws" library. You can install it with this command:
(And then continue with the step-by-step instructions as needed.)
Let's dive in and get started!

npm install ws

Get your free trial and API key

Head over to the tutorial that walks you through starting a free 14-day trial.
Once you have your API key, store it somewhere safe—you'll need it soon!

Setting Up a Node.js WebSocket Client to Receive Data

Find your NodeJS installation directory and create a new file called forexWsClient.js. Open it with your favorite code editor—Atom, VS Code, Notepad (on Windows), or any text editor on Linux.
Ready for some code magic? Let's dive in!

const WebSocket = require ('ws');
const ws = new WebSocket ('wss://marketdata.tradermade.com/feedadv');

First, we'll bring in the WebSocket object and set up a connection to TraderMade using their secure WSS URL. Once the connection is live, we'll send over our API key (the one you grabbed during the free trial signup) and the symbols we want to track. For example, let's focus on EURUSD.

ws.on('open', function open() {
     ws.send("{"userKey":"streaming_api_key", "symbol":"EURUSD"}");
  });

  ws.on('message', function incoming(data) {
    if(data != "Connected"){
            data = JSON.parse(data)
            console.log(data)
    }
  });

Running Your WebSocket Client

Save your forexWsClient.js file.
Open your terminal or command prompt and navigate to the file's location.
Run this command to fire up your client:

node forexWsClient.js

And That's a Wrap!

Congrats—you've successfully set up a Node.js WebSocket client to receive real-time Forex data! Now, sit back and watch the financial markets come to life with this smooth integration.

Transforming WebSocket Data into Minute-Based HLOC Bars with Node.js

Modules You'll Need

To make this script work, we'll rely on two key Node.js modules:

Fs: This is the File System module, which lets us read from and write to files.
Luxon: A handy JavaScript library designed for working with dates and times.

// Required Modules
const fs = require('fs'); // File System module
const { DateTime } = require('luxon'); // Luxon library for date-time formatting

File Paths

inputFilePath: This refers to the location of the input JSON file that holds the WebSocket data.
outputFilePath: This is where the processed HLOC data will be saved.

// File Paths
const inputFilePath = 'path/to/your/input/file.json'; // Update with your input file path
const outputFilePath = 'path/to/your/output/file.json'; // Update with your output file path

Reading Input Data

The script begins by asynchronously reading the input file using fs.readFile. It splits the file content into individual lines, each treated as a JSON object. This setup assumes that each line represents a separate WebSocket message in JSON format.

fs.readFile(inputFilePath, 'utf8', (err, data) => {
  if (err) {
    console.error('Error reading input file:', err);
    return;
  }

  let jsonData;
  try {
    jsonData = data.split('
').filter(line => line.trim() !== '').map(line => {
      try {
        return JSON.parse(line);
      } catch (error) {
        console.error('Error parsing JSON line:', error);
        return null;
      }
    }).filter(Boolean);
  } catch (error) {
    console.error('Error processing JSON data:', error);
    return;
  }

  if (!jsonData || jsonData.length === 0) {
    console.error('No valid JSON data found in the input file.');
    return;
  }

Grouping Data by Time Intervals

The groupDataByInterval function is key to organizing the data. It groups the WebSocket messages based on a set time interval—1 minute, to be exact. Here's how it works:

It starts by converting each data point's timestamp (ts) into a numerical value.
This timestamp is then divided by the interval duration (60,000 milliseconds), and the result is rounded down to determine which group (or interval) each data point belongs to.
All the data points that fall within the same interval are grouped.

const groupedData = groupDataByInterval(jsonData, 60000);
  const hlocData = calculateHLOC(groupedData);

  fs.writeFile(outputFilePath, JSON.stringify(hlocData, null, 2), (err) => {
    if (err) {
      console.error('Error writing output file:', err);

    } else {
      console.log('HLOC data written to', outputFilePath);
    }
  });
});

function groupDataByInterval(data, interval) {
  const groupedData = {};
  data.forEach(item => {
    const intervalKey = Math.floor(item.ts / interval) * interval;
    if (!groupedData[intervalKey]) {
      groupedData[intervalKey] = [];

    }
    groupedData[intervalKey].push(item);
  });
  return groupedData;
}

Calculating HLOC Data

Once intervals group the data, the calculateHLOC function takes over to compute the key HLOC values for each interval:

Open: The mid-value of the first data point in the interval.
High: The highest mid-value observed within the interval.
Low:The lowest mid-value seen in the interval.
Close: The mid-value of the last data point in the interval.

function calculateHLOC(groupedData) {
  const hlocData = [];
  for (const intervalKey in groupedData) {
    const intervalData = groupedData[intervalKey];
    const open = roundToFiveDecimalPlaces(intervalData[0].mid);
    const close = roundToFiveDecimalPlaces(intervalData[intervalData.length - 1].mid);
    const high = roundToFiveDecimalPlaces(Math.max(...intervalData.map(item => item.mid)));
    const low = roundToFiveDecimalPlaces(Math.min(...intervalData.map(item => item.mid)));

    hlocData.push({
      interval: Number(intervalKey),
      intervalDateTime: formatDateTime(Number(intervalKey)),
      open,
      high,
      low,
      close
    });
  }
  return hlocData;
}

Formatting and Writing Output

Once the HLOC values have been calculated for each interval, the data is organized into a well-structured JSON format, which includes:

Interval: A timestamp marking the start of each minute interval.
IntervalDateTime: The date and time corresponding to the interval timestamp, formatted for clarity.
Open, High, Low, Close: The calculated HLOC values. Finally, the processed HLOC data is saved to an output JSON file using fs.writeFile.

function formatDateTime(timestamp) {
  return DateTime.fromMillis(timestamp).toUTC().toFormat('yyyy-MM-dd HH:mm:ss');
}

function roundToFiveDecimalPlaces(value) {
  return Math.round(value * 1e5) / 1e5;
}
  } else {
    console.log('HLOC data written to', outputFilePath);
  }
});

Our output is all set!

{
    "interval": 1704453720000,
    "intervalDateTime": "2024-01-05 11:22:00",
    "open": 1.09152,
    "high": 1.09152,
    "low": 1.09132,
    "close": 1.09132
  },

  {
    "interval": 1704453780000,
    "intervalDateTime": "2024-01-05 11:23:00",
    "open": 1.09133,
    "high": 1.09153,
    "low": 1.09132,
    "close": 1.09152
  },

  {
    "interval": 1704453840000,
    "intervalDateTime": "2024-01-05 11:24:00",
    "open": 1.09153,
    "high": 1.09153,
    "low": 1.09141,
    "close": 1.09142
  }

Wrapping Up

This tutorial provides a simple and effective way to turn WebSocket data into valuable insights. By converting raw data into minute-based HLOC bars, developers and analysts can track market trends, spot patterns, and make well-informed decisions. The script's modular structure, combined with the Luxon library for handling dates and times, ensures both efficiency and accuracy in data processing and analysis.

Boosting WebSocket Scalability through a Python Proxy

Shridhar G Vatharkar — Fri, 10 Jan 2025 12:32:24 +0000

WebSocket protocol has revolutionized real-time communication over the web, making easy bidirectional conversations between clients and servers. Although WebSocket establishes persistent channels using the HTTP protocol and connection upgrade, scalability problems soon arise with rapidly growing applications. This article reveals how a simple Python-based proxy server for the WebSocket can more efficiently manage flow, ensuring superior performance and scalability.

The Scalability Problem in WebSocket Applications

The sheer volume of concurrent connections typically stands between most WebSocket applications and the light of day. A single server can quickly become overwhelmed with the client load, causing performance bottlenecks. A WebSocket proxy is a solution to the problem.

What Is A WebSocket Proxy?

A WebSocket proxy is a middleman between clients and servers, efficiently managing WebSocket traffic. It oversees the WebSocket handshake, distributes connections, and forwards messages to make communication smoother and reduce the load on the original WebSocket server.

Also Read: Scaling a Forex WebSocket with Python Proxy

Use Case: Scaling a Single WebSocket Source

Consider a real-time WebSocket source feeding live data, such as a financial market feed or chat system. Once the client base grows, this information must be distributed efficiently without overloading the source.

Python WebSocket Proxy

WebSocket Proxy Workflow

Handshake

Client connections to the WebSocket proxy. The proxy then manages the handshake and establishes the connection.

Load Balancing

The proxy distributes connections evenly across multiple instances of the WebSocket server so that no single server is overloaded.

Efficient Forwarding

WebSocket frames from clients are forwarded to the corresponding server instances for parallel processing, thus improving responsiveness.

Response Aggregation

The proxy collects responses from the servers and relays them back to the clients, ensuring transparency in the communication process.

Python Libraries for Implementation

Python libraries such as websockets and asyncio can be used to create a scalable WebSocket proxy. For detailed implementation, please refer to our in-depth tutorial on scaling WebSocket with Python.

Advantages of Python WebSocket Proxies

Scalability

It supports horizontal scaling, where the client base can be increased without problems.

Load Balancing

It uses intelligent algorithms to distribute connections, which prevent server overload.

Fault Tolerance

It redirects connections to healthy server instances during failures, which ensures uninterrupted service.

Conclusion

Introducing a WebSocket proxy is an effective way of scaling WebSocket applications. With its powerful libraries, such as websockets and asyncio, Python provides flexibility in implementing the solution. Managing live financial feeds, real-time chats, and other dynamic systems powered by WebSocket can significantly improve with a Python WebSocket proxy.

*Please go through the published initial tutorial on our website: *
Enhancing WebSocket Scalability with a Python Proxy

Implementing a Scalable Forex WebSocket Using a Python Proxy

Shridhar G Vatharkar — Tue, 31 Dec 2024 12:58:38 +0000

This guide will teach you how to create a WebSocket proxy server in Python.

Here's what the server will do:

Verify client identity: Before allowing clients to connect, it will check whether each has a unique "user key (API Key)."
Connect to another WebSocket: The server will connect to a separate WebSocket server.
Relay messages: The server will receive messages from the connected WebSocket and send them to all verified clients.

Before you begin:

Make sure you have installed Python 3.6 or a newer version. WebSockets need Python 3.6 or higher.
Install the WebSockets library: You can install it using the following command in your terminal.

pip install websockets

1. Getting Started

Create a new folder for your project.
Create a new Python file inside the folder and name it 'websocket_proxy_server.py.' This file will hold all the code for your server.

2. Create the WebSocket Server

Import the required libraries. You'll need the libraries you installed earlier.
Build the basic structure of your server. Use the WebSockets library to create the foundation for your server.

import asyncio
import websockets
import json

class WebSocketProxy:

    def init(self, source_url, symbols):

        self.source_url = source_url
        self.clients = set()
        self.symbols = symbols
        self.valid_user_key = "yourValidUserKey"  # Single valid user key for authentication

    async def on_open(self, ws):

        print("Connected to source")
        symbols_str = ",".join(self.symbols.keys())
        init_message = f"{{"userKey":"your_api_key", "symbol":"{symbols_str}"}}"
        await ws.send(init_message)

3. Connect and Verify Clients

Ensure that the server is all set to accept connections from clients.
Add a check to verify each client's identity. As a client tries to connect, the server should ask for a "user key." Only clients with the correct key will be allowed to connect.

async def client_handler(self, websocket, path):

        try:

            # Wait for a message that should contain the authentication key
            auth_message = await asyncio.wait_for(websocket.recv(), timeout=10)
            auth_data = json.loads(auth_message)
            user_key = auth_data.get("userKey")

            if user_key == self.valid_user_key:
                self.clients.add(websocket)
                print(f"Client authenticated with key: {user_key}")

                try:
                    await websocket.wait_closed()

                finally:
                    self.clients.remove(websocket)

            else:

                print("Authentication failed")
                await websocket.close(reason="Authentication failed")
        except (asyncio.TimeoutError, json.JSONDecodeError, KeyError):
            print("Failed to authenticate")
            await websocket.close(reason="Failed to authenticate")

4. Connect to the Source and Share Messages

Create a function that keeps the server connected to the original WebSocket.
This function should automatically send messages received from the original WebSocket to all successfully verified clients.

async def source_handler(self):
        async with websockets.connect(self.source_url) as websocket:
            await self.on_open(websocket)
            async for message in websocket:
                await self.broadcast(message)

    async def broadcast(self, message):
        if self.clients:
            await asyncio.gather(*(client.send(message) for client in self.clients))

5. Start the Server

Create a function to start the server and listen for connections.
Add code to run this function, starting your WebSocket proxy server.

def run(self, host="localhost", port=8765):
        start_server = websockets.serve(self.client_handler, host, port)
        asyncio.get_event_loop().run_until_complete(start_server)
        asyncio.get_event_loop().run_until_complete(self.source_handler())
        asyncio.get_event_loop().run_forever()

if name == "main":
    symbols = {"EURUSD": {}, "GBPUSD": {}, "USDJPY": {}, "AUDUSD": {}, "USDCAD": {}}
    source_url = "ws://example.com/source"
    proxy = WebSocketProxy(source_url, symbols)
    proxy.run()

In Summary

You have successfully developed a Python-based WebSocket proxy server. This server can authenticate client identities, maintain a persistent connection to a designated data source, and effectively distribute messages received from the source to all verified clients. This functionality proves invaluable for applications that necessitate the secure and instantaneous dissemination of data from a singular origin to a diverse user base.

Next Steps

Thorough server testing is crucial to ensure optimal performance and reliability. It verifies its proper handling of connections and message transmission. To enhance efficiency, consider implementing load-balancing mechanisms and customizing connection headers. Finally, it is advisable to deploy the server to a suitable environment for production deployment, such as a cloud service specifically designed to accommodate long-term network connections.

Also, please take a look at the originally published tutorial on our website: Scaling a Forex WebSocket with Python Proxy