Forem: Blankly

Build an LSTM Neural Network Bot for Trading

Brandon Fan — Wed, 06 Apr 2022 20:24:09 +0000

Today's Model

Hey there everyone. My name's Aditya, a developer advocate here at Blankly.

Today, we'll look at using the Blankly package to build a basic machine learning model for trading. Machine learning has been one of the hottest trends in algotrading during the last few years, as we've only recently reached the level of computing power and amount of data needed to build and train successful models. This specific model we will be an LSTM (Long Short Term Memory) Neural Network, which is a type of neural network that stores a "memory", allowing it to incorporate past data passed into the model into future predictions. This structure makes LSTMs great for sequential data, like stock prices.

To build the model, we'll use PyTorch, a popular library for Machine Learning. Luckily, PyTorch handles most of the hard work for us -- backpropagation, gradient descent, and layer structure, to name a few examples -- so we can focus on building the actual model.

LSTMs

The first question you might have is what is an LSTM? As we said above, LSTMs incorporate a "memory" into their model structure, but that explanation is a bit vague. To be more precise, LSTMs compute multiple functions at every stage:

Without getting too technical, $h_t$ is the "hidden state" --- the "memory" of the model, $f_t$ is the "forget gate", $i_t$ is the "input gate", $o_t$ is the output gate, $\tilde{C}_t$ is the candidate cell state, and $C_t$ is the cell state. At every step, the input to the model gets passed into the input gate, influences the hidden state, and potentially changed the cell state. The decisions for how each input affects the rest of the model are decided by neural network layers, which are the weights we train using gradient descent.

For their ability to dynamically include data in the past in current models, LSTMs are one of the most popular models for stock prediction. There are countless technical indicators for financial markets, and so using LSTMs allows us to dynamically choose which data from which time frame is most relevant to predicting future prices.

Stock Market Predictions with LSTM in Python - DataCamp

Machine Learning to Predict Stock Prices - Data Science

Using Keras LSTM Model to Predict Stock Prices - KDNuggets

Things to Watch Out For

There are definitely pitfalls to assuming LSTMs automatically lead to success. For one, prediction accuracy is limited by the available training data, and our training set is fairly limited and basic. However, in general, short-term stock price data has an element of randomness, so the accuracy of LSTM predictions has is limited. For this reason, we'll use an average of three predictions -- in this way, we'll reduce inaccuracy due to randomness, and so improve overall results.

Overview

First, we'll initialize all the elements of our environment -- the keys for the API we need, as well as the imported libraries we need. Then, we'll process the input data we have and convert into a format that is appropriate for model training. Then, we'll run through our training loop and find the optimal weights for our model. After that, we'll find a way to convert output signals from our model into decisions onto whether to buy and sell, along with how much, and use those to define a strategy. Finally, we'll backtest this strategy and analyze its performance to determine whether to use it for live crypto trading or to improve it further.

Initialization

We'll initialize the basics of our Blankly environment with the command blankly init. Once done, we get template .json files that we'll need for configuring backtests. Most importantly, we'll need to input our API keys into keys.json.

$ blankly init

Here are our imports. We’ll import blankly (of course), NumPy for use with preparing our data, and some PyTorch utilities we’ll need for our model

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.nn import LSTM
import torch.optim as optim
from torch.autograd import Variable

Data Preparation

This method will help us split the data into “input/output”. We only actually have one sequence of data — all the prices from the past year. However, we can generate smaller “episodes” by

Taking all consecutive periods of length seq_length
Splitting off the last output_size values and putting those into one array
Leaving the first seq_length - output_size values into another array.

In the end, we’re left with enough data to run training on. For this model, we’ll use seq_length = 8 and output_size = 3, but these are somewhat arbitrary. This gives us training sequences of 8 - 3 = 5 data points to predict the sequences of length 3 from. The idea is that we'll train the model on those sequences of 5 data points, end up with a model that can predict the price movements for the next 3 days to solid accuracy, and then average the price movements over a 3-day period to obtain a final prediction for the price movement on a day. Then, we'll use our prediction to decide whether to buy our sell -- if our model tells us the price is likely to increase tomorrow, we'll buy, and if the model tells us the price is likely to decrease, we'll sell.

def  episode_gen(data, seq_length,output_size):
    x = []
    y = []
    #Loop through data, adding input data to x array and output data to y array
    for i in  range(len(data)-seq_length):
        _x = data[i:(i+seq_length - output_size)]
        _y = data[i+seq_length - output_size:i + seq_length]
        x.append(_x)
        y.append(_y)

    return np.array(x),np.array(y)

Model Inputs and Feature Engineering

Here, we’ll do the majority of work on our model.

To start, we’ll pull the data we need. Blankly comes with built in indicators, so we can input our historical price data into those functions to get indicators out. For this example, we’ll use RSI and MACD. RSI stands for Relative Strength Index, and is a measure of momentum. Its formula is

where we look at the data over the past 14 days. Typical overbought/oversold levels for RSI are 70 and 30 (respectively)

MACD stands for Moving Average Convergence Divergence. The MACD is calculated by subtracting the 26-period exponential moving average (EMA) from the 12-period EMA. We then calculate the MACD signal line by taking the 9-period EMA of the MACD.

Implementing these functions by hand is a bit tricky, but fortunately, Blankly has indicators built in.

def init_NN(symbol, state: blankly.StrategyState):

    interface = state.interface
    resolution = state.resolution
    variables = state.variables

# Get price data

    variables['history'] = interface.history(symbol, 300, resolution, return_as='list')['close']

'''We use Blankly's built-in indicator functions to calculate indicators we can use along with price data to predict future prices
'''
    rsi = blankly.indicators.rsi(state.variables['history'])
    macd = blankly.indicators.macd(state.variables['history'])

'''We'll break the historical Ethereum data into 8 day episodes

and attempt to predict the final three days using the first 5.

'''
    seq_length = 8
    output_size = 3

We also do a bit of feature engineering. The first step is to convert our price data into daily increases by dividing each day’s price data by the day before. This helps in training by “scaling” the data into the same range. We’ll also limit the range of data to the range in which we have data on all indicators — day 26 and on.

'''Feature engineering -- here, we calculate the price change from the day before
as a ratio. This is useful because it means we have less issues with scaling with the model,
as the data will all already be in roughly the same range. We ignore the first 25 elements
because we want every observation to have corresponding RSI + MACD data, and MACD requires
26 periods.
'''

    x = [variables['history'][i] / variables['history'][i-1] for i in  range(25,len(variables['history']))]
    x, y = episode_gen(x, seq_length, output_size)
    y = Variable(torch.Tensor(np.array(y))).unsqueeze(0)

#RSI data gathering
    x_rsi = rsi[11:]
    x_rsi,_ = episode_gen(x_rsi,seq_length, output_size)

#MACD data gathering
    macd_vals,_ = episode_gen(macd[0], seq_length, output_size)
    macd_signals,_ = episode_gen(macd[1],seq_length, output_size)

In this loop, we’ll feed the data we’ve collected into an array of dimensions num_episodes $\times$ output_size - seq_length $\times$ 4

'''In this section, we put all the features we just extracted into one NumPy array
which we then convert to a PyTorch tensor that we can run our model on.

'''

    x_agg = np.zeros((len(macd_signals),seq_length-output_size, 4))

    for i in  range(len(macd_signals)):
        for j in  range(seq_length - output_size):
            x_agg[i][j][0] = x[i][j]
            x_agg[i][j][1] = x_rsi[i][j]
            x_agg[i][j][2] = macd_vals[i][j]
            x_agg[i][j][3] = macd_signals[i][j]
    x_tot = Variable(torch.Tensor(x_agg))

Training

Now, it’s time to train. We’ll use a mean-square-error loss function and an Adam optimizer for a model that’ll take the price data into an LSTM, output that to a linear layer with sigmoid activation, and add a constant 0.5. We do this because the sigmoid function returns 0.5 when the input is 0, so we effectively train our model layers to output positive values for an increase and negative values for a decrease.

In terms of hyperparameters, our only significant ones here are the 10,000 epochs and learning rate of 0.0003. These came through running through a couple of backtests with grid search. Lower learning rates generally result in more accurate models, with the caveat that this accuracy requires longer training times to achieve. The 10,000 epochs number was chosen as a balance between performance and speed. To get an idea for training times, we include a line that'll print out the loss every 500 epochs, ultimately allowing us to see when the loss stabilizes, indicating convergence. Together, our results show that training our model for 10000 epochs with 0.0003 learning rate results in a loss value that stabilizes just before we finish, along with a low value for our loss-function itself.

More sophisticated methods than manual search exist, and should be used in several cases; for example if we were to try to adapt a type of architecture to a set of tickers and couldn't afford to find the ideal hyperparameters by hand for training the model on each ticker's data. Methods like Bayesian Hyperparameter Optimization exist that automatically search for optimal hyperparameters, using results of past trials to guide future ones. However for this example, manual search is good enough and keeps the attention on the main model features and attributes.

num_epochs = 10000
learning_rate = 0.0003

state.lstm = LSTM(4,20, batch_first = True)
state.lin = nn.Linear(20,3)

criterion = torch.nn.MSELoss() # mean-squared error for regression

#Optimizer: make sure to include parameters of both linear layer and LSTM

optimizer = torch.optim.Adam(
{'params': state.lstm.parameters()},
{'params': state.lin.parameters()}
], lr=learning_rate)

# Train the model
for epoch in  range(num_epochs):

    #run model
    outputs, (h_n, c_n) = state.lstm(x_tot)
    out = state.lin(h_n)
    out = 0.5 + F.sigmoid(out)
    optimizer.zero_grad()

    loss = criterion(out, y) #calculate loss function
    loss.backward() #backprop
    optimizer.step() #gradient descent



    #Output loss functions every 500 epochs so we can make sure the model is training

    if epoch % 500 == 0:
        print("Epoch: %d, loss: %1.5f" % (epoch, loss.item()))



'''We use this in the trading algorithm for more stability.

Essentially, instead of relying on a single output of the model

to tell us whether to buy or sell, we average the readings from three different calculations

(3 days before, 2 days before, day before)

'''
state.lastthree = [[0,0],[0,0],[0,0]]

From Model to Strategy

Now that we’ve trained our model, we need to use its outputs in a price event to backtest. First, we extract data and indicators from the last 5 days and put them into a tensor of dimension 1 $\times$ seq_length - output_size $\times$ 4.

def price_lstm(price,symbol,state: blankly.StrategyState):
    state.variables['history'].append(price) #Add latest price to current list of data

    '''Here, we pull the data from the last few days, prepare it,
    and run the necessary indicator functions to feed into our model

    '''
    into = [state.variables['history'][i]/state.variables['history'][i-1] for i in  range(-5,0)]
    rsi = blankly.indicators.rsi(state.variables['history'])
    rsi_in = np.array(rsi[-5:])
    macd = blankly.indicators.macd(state.variables['history'])
    macd_vals = np.array(macd[0][-5:])
    macd_signals = np.array(macd[1][-5:])

    '''We put the data into the torch Tensor that we'll run the model on

    '''
    pred_in = np.zeros((1,len(into),4))

    for i in  range(len(into)):
        pred_in[0][i][0] = into[i]
        pred_in[0][i][1] = rsi_in[i]
        pred_in[0][i][2] = macd_vals[i]
        pred_in[0][i][3] = macd_signals[i]
    pred_in = torch.Tensor(pred_in)

Then, we run our model on the tensor, giving us predictions for the next three days.

'''Run the data through the trained model.
The field out stores the prediction values we want
'''

out,(h,c) = state.lstm(pred_in)
out = state.lin(h)
out = 0.5 + F.sigmoid(out)

We generate our prediction for a days move by looking at the predictions from 3,2, and 1 days before and averaging them.

'''This definitely could be shortened with a loop,
but basically, we add the percentage increase to the other values in the
3-day-average array. We also increment a counter showing how many values have been
added before averaging. This handles the edge case of the first few values (where
we wouldn't divide by 3)
'''
state.lastthree[0][0]+=out[0][0][0]
state.lastthree[0][1]+=1
state.lastthree[1][0]+=out[0][0][1]
state.lastthree[1][1]+=1
state.lastthree[2][0]+=out[0][0][2]
state.lastthree[2][1]+=1
'''The avg price increase is calculated by dividing the sum of next day predictions
by the number of predictions for the next day.
'''

priceavg = state.lastthree[0][0]/state.lastthree[0][1]

Finally, we implement our buying and selling logic. If our prediction is 1 — that is, we think tomorrow’s price will be greater than today’s, we buy an amount proportional to how far above 1 the prediction is. If we think tomorrow’s price will be less, we sell an amount proportional to how far below 1 the prediction is. There are many different ways to choose buy/sell sizing amounts, some with more mathematical backing, but this works fine to just demonstrate an example and is fairly simple.

curr_value = blankly.trunc(state.interface.account[state.base_asset].available, 2) #Amount of Ethereum available
if priceavg > 1:
    # If we think price will increase, we buy
    buy = blankly.trunc(state.interface.cash * 2 * (priceavg.item() - 1)/price, 2) #Buy an amount proportional to priceavg - 1
    if buy > 0:
        state.interface.market_order(symbol, side='buy', size=buy)
elif curr_value > 0:
    #If we think price will decrease, we sell
    cv = blankly.trunc(curr_value * 2 * (1 - priceavg.item()),2) #Sell an amount proportional to 1 - priceavg
    if cv > 0:
        state.interface.market_order(symbol, side='sell', size=cv)

state.lastthree = [state.lastthree[1], state.lastthree[2], [0,0]] #Shift the values in our 3-day-average array

Backtest

To actually backtest, we’ll need to connect to an API. I used FTX, but Blankly also currently supports Alpaca, Binance, Coinbase Pro, KuCoin, and OANDA. We then create a Blankly Strategy, add our price event and initialization, and run!

exchange = blankly.FTX() #Connect to FTX API
strategy = blankly.Strategy(exchange) #Initialize a Blankly strategy
strategy.add_price_event(price_lstm, symbol='ETH-USD', resolution='1d', init=init_NN) #Add our price event and initialization
results = strategy.backtest(to='1y', initial_values={'USD': 10000}) #Backtest one year starting with $10,000
print(results)

Results:

Blankly Metrics:
Compound Annual Growth Rate (%): 98.0%
Cumulative Returns (%): 97.0%
Max Drawdown (%): 20.0%
Variance (%): 12.59%
Sortino Ratio: 2.17
Sharpe Ratio: 1.5
Calmar Ratio: 2.69
Volatility: 0.35
Value-at-Risk: 395.43
Conditional Value-at-Risk: 10.0
Risk Free Return Rate: 0.0
Resampled Time: 86400.0

We find a CAGR of 98% -- very good, along with a Sharpe Ratio of 1.5 and a Sortino Ratio of 2.17, both also considered strong. However, the first successful backtest is only the first step in deploying a profitable strategy. Before deploying, we need to test much more thoroughly: with different data (other coins, possibly stock tickers), for different times (this backtest only captures one time period -- what about other periods in the bearish/bullish market cycle?), and for robustness with regard to randomness. While good, the results we obtain are still in the range of "lucky", and so we need to test until we're highly confident that our model will profit.

There are many ways we could improve this strategy -- deepen the machine learning model, pull more/better input data, or better choose amounts to buy/sell, but this is a very strong start, and Blankly's package makes it very easy to edit and test this model. If you're interested, a GitHub repository containing a full Python Notebook is here.

If you're interested in learning more, talk to us here @ Blankly and check out our open source package. We'd love to chat! We're constantly in our Discord too!

Build a Backtester in Python in 10 Minutes

Brandon Fan — Wed, 06 Apr 2022 20:03:05 +0000

Welcome to the World of Algotrading

So you’ve made the leap of faith now to discover algotrading. It’s time to get up and running by building your first trading algorithm. You’ve joined the ranks with the many fintech companies and hedge funds that run millions of backtests behind the scene. Backtests are our way of understanding what our algorithms are doing: how they’re making decisions, determining how they invest, and ultimately how they make money. Yet the countless pitfalls in running a backtest like survivorship bias, look-ahead bias, and many other biases, lack of data, and more, can oftentimes lead to confusion. This tutorial will walk you through how to build two backtesters that mitigate these issues: one of which utilizes a vector-based approach, and the other an event-driven approach (don’t worry we’ll go into more detail). We’ll also show you where these backtesters fail, and why building a “fully-production ready” one might be a little harder than you think.

The Vectorized Backtester

What is Vectorization?

Let’s take a look at this first backtester, a vectorized one. First, what is a vectorized backtest? The best way is to prescribe an example. Let’s say I wanted to build a trading algorithm that looked at two moving averages and identified when they crossed (hmmm...sounds like the golden cross). So I download some price data and proceed to calculate the moving averages. Luckily, since it’s a static dataset (since we’re resting), I’m actually able to “precompute” my moving averages. That means that I’m able to compute the 50-day SMA and the 200-day SMA almost immediately. And then figure out when they cross by simply doing a boolean comparison using pandas. From there, I can calculate all I need (my overall account value, PnL) and ultimately my backtest. All vectorized — no for loops required, and we can use linear algebra and matrices to make the computation extremely fast. That’s the whole idea of vectorization. Rather than computing them step by step through time (like using a for loop), we precompute our trading algo (our signal in this case is the cross), and then calculate everything using linear algebra. Let’s code this up in practice.

Quick Backtester Spec

What is Our Goal: Our goal is to build a backtester that is able to take in a signal that can be run across price data and then calculate the total account value returns over time.

Process: We will precompute the signal and then use the signal results to then calculate our Profit & Loss and our account values over time.

What Technologies: We will use pandas and numpy to do everything and get data from yfinance

Step by Step Implementation

Imports:

import pandas as pd
import numpy as np
import yfinance as yf

Pandas and NumPy are both useful Python libraries for manipulating data. YFinance is a library allowing us to access price data from Yahoo! Finance. YFinance outputs data in the form of Pandas DataFrames, but NumPy objects are often more convenient to work with, so we’ll use both.

Getting Data:

spy = yf.Ticker('SPY')
hist = spy.history(period = '2y')
pers = hist.loc[:,'Close'].apply(np.log).diff(1)

YFinance works through Ticker objects, which collect all the data for a single ticker. As you can guess, our first line does this for the S&P 500. Our second line sets the timeframe we’re looking over — 2 years. The third line helps us work with linear algebra. We take the log of prices so that we can add differentials from day to day and then exponentiate to get the overall return. Diff(1) takes the difference between each pair of elements, allowing us to consider the day-to-day returns, which can be easily combined to represent buying and selling at different times.

Signal Calculation

def sma(prices, period):
    return prices.loc[:,'Close'].rolling(window=period).mean()

We define a helper function here that returns a Pandas DataFrame that contains the specified moving average for each day.

sma50 = sma(hist,50).fillna(0).shift(periods=1, fill_value=0)
sma50[:200] = 0
sma200 = sma(hist,200).fillna(0).shift(periods=1,fill_value=0)
signal = sma50 - sma200
sig = signal.apply(np.sign)

We define our signal — the golden cross. After calculating each moving average, we zero elements in the first 200 days, where the SMA200 isn’t defined. We then take the sign of each, so that our signal tells us to go long when SMA50 is over the SMA200 and go short when the SMA50 is under, staying out when the two are equal.

Backtest

returns = sig * pers
print(returns.cumsum().apply(np.exp))

We run the signal over the log-prices and exponentiate to get our final values of the backtest. Our final printed value is 1.188356, corresponding to a gain of nearly 19% .

The Event-Driven Backtester

What is Event-Driven?

Okay, so we saw the vectorized backtest. Unfortunately, as we know it, we can’t actually use vectorization in practice. Why? Well we don’t know all the prices beforehand, so we simply can’t precompute our entire signal and then execute on it through time. And that’s the main issue with vectorized backtest: what we gain in speed, we sacrifice in realistic simulation. ****This is why we introduce an event-driven framework. The event-driven framework instead loops through time to ultimately simulate a trading algorithm directly trading it through each time step (i.e. a for loop). This reduces the potential for biases like look-ahead by only allowing specific data to be accessed at one time. Going back to our golden cross example, this means that we would be computing our moving averages as we go.

Quick Backtester Spec

What is Our Goal: Our goal is to build a backtester that is able to take in a signal that can be run across price data and then calculate the total account value returns over time.

Process: We will create an event-driven backtester that will loop through as series of price data and will compute the signal as we move forward in time.

What Technologies: We will use pandas and numpy, some looping, and we’ll get data from yfinance

Step by Step Implementation

Imports

We’ll use the exact same imports as last time.

Getting Data

We’ll get data in the same way as before

Data Preparation

prices = hist.loc[:,'Close'].to_numpy()
running_sma50 = prices[150:200].mean()
running_sma200 = prices[:200].mean()

We convert our data to NumPy for easy slicing and calculate our running SMAs for a start to trading on day 200.

Event-Driven Backtesting Loop

event = 0
rets = [0]
for i in range(200, hist.shape[0]):
    rets.append(rets[-1] + (event * (pers.iloc[i])))
    running_sma50+=(prices[i] - prices[i-50])/50
    running_sma200+=(prices[i] - prices[i-200])/200
    event = np.sign(running_sma50 - running_sma200)

Like before, we track the operation to do on a given day — buy (1), short sell (-1), or stay neutral (0), and we store it in the variable event. Our array rets tracks log returns. Each day we have data for (from 200 til the end), we first take the action calculated by the signal on the previous day, execute it, and append to rets. Then, we update the running SMAs by adding the most recent price point, subtracting the one that should be dropped (from 50/200 days ago), and dividing that difference by the appropriate period. Finally, we calculate the appropriate operation by taking the sign of the difference between SMAs.

Output

print(np.exp(rets))

Once we exponentiate, we’re left with the cumulative returns. Like before, we get the 1.19 value, corresponding to an ~ 19% gain. While the vectorized backtesting may have been faster, we had sacrificed flexibility. Real trading algorithms often have to dynamically allocate capital, something that vectorized backtesting often doesn’t allow. Although an event-based backtester does address some of these concerns, it is far from perfect.

Limitations of Both Approaches

Ok sweet—We were able to build out a backtester that was able to output the exact results that we wanted, a graph of our account values over time and our calculate Profit & Loss. If you’re even more stellar, you can also take a look at this article ←- link here that walks you through how to add in the top backtesting metrics that you would like to measure.

As expected, the quick backtesters that we’ve made aren’t perfect. There are many nuances that our basic backtesters are not able to address, and we go over a few below.

What if I wanted to build out a new trading algorithm?

We built out these backtesters to work specifically for this trading algorithm—a golden cross. Now imagine if we had to build the same set of code again for another type of trading algorithm. What if the new trading algorithm wanted to incorporate some additional data that wasn’t price? How about if it uawa future data, or orderbook data? We would have to rewrite all of this code again to support the new data formats, and handle the necessary connections needed for new types of algorithms. Oftentimes, developing a data pipeline can be huge timesavers when building your own backtesting infrastructure.

What if I wanted to test multiple algorithms or tickers at once?

Most of the time, we’re not putting all of our eggs into one basket, we’re building multiple algorithms at one time and creating a portfolio of them. Better yet, we run the algorithm on multiple tickers and datasets. Now this becomes even more complicated, do you loop through all of once, do you loop through each separately? What if the algorithms had to communicate with each other? How do you manage things like state? What if one was on a crypto exchange and the other was using stocks? With the current design of our backtester, it would take a lot of work and modification to get it to the point where it’s flexible enough to model these types of algorithms.

Are these backtests really representative of the real world?

With the backtests that we’ve made, are we truly modeling everything out? How about things like commission fees in crypto. What about margin accounts and margin interest? What about slippage and accurately modeling order flow? These are concerns that our backtest simply do not handle and don’t provide an idealistic way of looking at all the data to ultimately and accurately represent reality. These are extremely important especially as you start trading more and more money.

Introducing a Production Ready Backtester

Because of these many concerns, though it’s good to build out your own, oftentimes it’s better to utilize a backtester that mitigates all of the limitations that we’ve seen above. There are a ton of different ones including backtesting.py, backtrader , bt, vectorbt, and more. But some things that you have to consider too is that when you’re done with backtesting, how are you going to deploy your trading algorithm live? Do you want to rewrite all of the code again (that’s what current systems and frameworks do and that’s also what you’ll have to do with the backtester above). That’s why we created blankly. Blankly is the only package on the market that allows you to go from a trading idea, backtesting it in a realistic environment, paper trading on any exchange, and deploying live on optimized infrastructure in minutes. All you have to do is create .backtest(), and switch it out when you’re ready to deploy and launch. Check out our example for the golden cross right here ←linked

from blankly import Alpaca, Interface, Strategy, StrategyState
from blankly.indicators import sma

def init(symbol, state: StrategyState):
    interface: Interface = state.interface
    resolution: float = state.resolution
    variables = state.variables
    # initialize the historical data
    variables["history"] = interface.history(symbol, 800, resolution, return_as='deque')["close"]
    variables["has_bought"] = False

def price_event(price, symbol, state: StrategyState):
    interface: Interface = state.interface
    # allow the resolution to be any resolution: 15m, 30m, 1d, etc.
    resolution: float = state.resolution
    variables = state.variables

    variables["history"].append(price)

    sma200 = sma(variables["history"], period=20)
    sma50 = sma(variables["history"], period=10)[-len(sma200):]
    diff = sma200 - sma50
    slope_sma50 = (
        sma50[-1] - sma50[-5]
    ) / 5  # get the slope of the last 5 SMA50 Data Points
    prev_diff = diff[-2]
    curr_diff = diff[-1]
    is_cross_up = slope_sma50 > 0 and curr_diff >= 0 and prev_diff < 0
    is_cross_down = slope_sma50 < 0 and curr_diff <= 0 and prev_diff > 0
    # comparing prev diff with current diff will show a cross
    if is_cross_up and not variables["has_bought"]:
        interface.market_order(symbol, 'buy', int(interface.cash / price))
        variables["has_bought"] = True
    elif is_cross_down and variables["has_bought"]:
        interface.market_order(
            symbol, 'sell', int(interface.account[symbol].available)
        )
        variables["has_bought"] = False

if __name__ == "__main__":
    alpaca = Alpaca()
    s = Strategy(alpaca)
    s.add_price_event(price_event, "MSFT", resolution="1d", init=init)
    s.add_price_event(price_event, "AAPL", resolution="1d", init=init)
    s.backtest(initial_values={"USD": 10000}, to="2y")

What’s even more awesome is that this code can in one line be switched to production, without ever having to rewrite it.

Conclusion

Backtesting is a critical part of the algotrading journey and it’s absolutely critical to get it right. Though the backtesting results never imply future performance, it sure as heck can represent the future....if you build it right. In this article, we walked through how to build two types of backtesters and the limitations of each and walked through a couple of other examples.

Why Building a Trading Algorithm is More Than Just the Algorithm - 3 Things

Brandon Fan — Sat, 19 Feb 2022 21:21:31 +0000

Simulations are NOT the Real World

The Problem

It’s super easy to get up and running with code. With the rise of data science as a field, datasets are far and wide. Accessible from just about any venue. Take a look at Kaggle, QuiverQuant, Yahoo Finance, or even directly from the brokerages and exchanges. Developers can easily download data directly as a .csv or .json and quickly get up and running by utilizing frameworks like backtesting.py or vectorbt. “Great, it seems like I can get up and running and I’ll have an awesome money making trading algorithm in no time”.... unfortunately, wrong. Why is this wrong? Well, simulation is NOT the real world. The real world is not a CSV file—the real world is a stream of events. Cause and effect. The real world works in a fashion where new data comes in, you make a decision, and then you figure it out, not “I have all of this data, let me run this all through time and figure it out”. Indeed, the data sources that you get in real-time are almost completely different from the data sources you use in simulation. Rather than .csv you use WebSockets; rather than QuiverQuant you use APIs; rather than backtesting frameworks you use more robust, event driven packages. Without it, you’re stuck duplicating code, rewriting it into an event-based system, and ultimately using that to go into production, and who knows if your code is going to change along the way.

The Solution

Use an Event-Driven Package

As mentioned, focus on building trading algorithms with the production world in mind. It is perfectly fine to utilize a vectorized backtesting system for rapid iteration, but be prepared to have to rewrite your code to put it into a paper trade or live environment. An event-driven backtesting system can provide much more power and flexibility.

As by design, it can be integrated for both historical backtesting and live trading without compromising the overall structure and logic. As vectorized backtesting requires all data to be available at once to perform any sort of analysis, this is not possible, of course, in a live environment, with no data of the future at hand.
It can prevent look-ahead bias. An event-driven backtester will never factor future data—instead, it takes each point of the most recent market data as an event to act upon.
Event-driven backtesters provide customization in market orders. From basic market to limit orders, to market-on-open and market-on-close orders, stop orders, and more, they can all be built in as a custom handler.

Focus on Figuring Out Your Live Data Sources Early

Again, it’s super easy to think that you’ll have all the data in the world in the real environment. Most of the time, the data you have in testing is very different from the data that you receive from the real environment—and ensuring that you’re able to match up the two, easily format the data into the same way is going to be super super important. Figuring out where your historical datasets come from and corollary live data streams will come from will be imperative.

So uh....where am I going to make trades?

The Problem

Let’s take, for example, the crypto world. In this space, there are over ten difference crypto exchanges all with their different ploys and different opportunities, including Binance, Coinbase, FTX, Kraken, KuCoin, and much, much more. Some originate in Fiat, others originate in crypto. Moreover, what if you wanted to switch your trading algorithm to run on forex or stocks? The list of exchanges increase by the dozen, and each one is its own independent entity. These kinds of questions, though a little less important in a backtesting environment, become imperative in a live environment. In order to make trades, you HAVE to connect to an exchange or brokerage... there’s no way around it. For many, the connection to an exchange is completely skipped over during the testing and experimental stage. Again, de facto-ing to downloading datasets is great, but how will you operate on the logic of making trades? How are you going to switch your algorithm into a paper trade environment for every exchange? These are critical questions to ask yourself as you implement your trading algorithm: what asset are you going to use, what exchange are you going to integrate with, and how are you going to actively manage and maintain that exchange as it changes its API (look at Alpaca, they’ve been constantly changing their APIs and systems).

The Solution

For all the DIYBIY people out there, this is probably one of the largest pain points. You have to actively manage exchange connections while also maintaining your code and ensuring that your strategy is always updating and improving. If you know that you’re going to just be using one exchange or asset type, this is relatively easy as there are typically wrappers in various languages for each exchange. However, if you are not really sure and you plan on changing, choosing a cross-asset and cross-exchange package such as CCXT that allows you to test and deploy between multiple with ease is a game-changer. The worst case scenario you let your model run and the exchange changes their API, so offset this risk to someone else so you can focus on building a better trading algorithm.

What’s my deployment infrastructure?

The Problem

This is the biggest whammy of them all. How are you going to launch and deploy your trading algorithms? Now that you have code that works, and assuming that you have solved problems 1 and 2, you’re going to have figure out problem 3. Are you going to choose AWS or GCP? Are you going to choose a cluster near the exchange? Are you going to SSH into the EC2 instance? What about DigitalOcean? Do I use a cloud run instance or do I use a lambda function? The number of decisions that you have to make just to deploy your trading algorithm is a whole new skillset of cloud that you don’t want to worry about. Don’t want to use the cloud? How are you going to actively manage firewall connections, security, ensuring that your computer is always on, that your desktop is running. How can you make sure your wifi doesn’t cut out? All of these questions are important considerations as you begin moving deeper into automated trading.

Again, what was your goal at the beginning? It was to make sure that we had an automated system that can make trades; NOT a system that we have to constantly worry about.

The Solution

There really isn’t any solution around this. The best way to do this is to deploy onto the cloud into an optimized environment. Yet managing CPU, RAM, and other issues like monitoring, SSH’ing, configuring security for API keys, ensuring isolation and proper firewall connections are going to be a major pain.

So, how can we help?

Building a trading algorithm is not just about building a strong strategy. Though that is a precursor to everything, not building it the proper way can lead to major complications, especially in production. Imagine spending 2 months building out an optimized algorithm and an optimized system only to realize that it won’t work in production. That’s how Blankly came to existence—we were looking for the best solution possible to put together each of these problems, and ultimately take away the backend, infrastructure side of making trading algorithms.

With the Blankly Package to help with maintaining active exchange connections and optimized event-driven backtesting, and the Platform to deploy your algorithms and monitor your strategies live in the cloud, your quant workflow is sped up from head to toe. The barrier to entry shouldn’t be around infrastructure and data, and should be around how to use data and algorithms to make the most profit. Let us do the heavy lifting, so you can focus on building better algorithms.

Want to learn more? Check out our website, blankly.finance, or reach out to us by email (hello@blankly.finance) or our Discord.

Why You Shouldn’t Use a No-Code Trading Platform - 3 Reasons

Brandon Fan — Mon, 07 Feb 2022 02:48:27 +0000

Growing Trend of Algo-Trading

There’s a huge buzz in the market nowadays that “anyone can make money” in the markets. True, it’s gotten so much easier, and with the advent of technologies of all alike, it sure has hell has made investing and opportunities in the market so much more formidable. With the rise of Crypto and decentralized finance alone, new asset classes have been born including perpetual futures, liquidity pools, staking protocols, and yield farming strategies that go against traditional markets, offering new hedges and opportunities for those that can take advantage...and that’s the key, for those who can take advantage. When we look at investing, there are a multitude of things that we care about: risk management, asset control / custody, security, volatility, and ultimately returns.

With all of this in mind, as we look at the world of quant and trading, we see really two stark paradigms. The first is the DIYBIY (do it yourself, build it yourself) people who build just about everything in-house from the underlying infrastructure, to the exchange connections, to the live monitoring and more (typically around $10M+ average spend from hedge funds). Finally, the other side of the paradigm is the no-code trading platforms, the “we build everything completely for you”, and “click a few buttons, and you’ll be able to build your own trading algorithm in no time”. Now this is a paradigm, and of course there are middle grounds, but here’s 3 reasons you do not want to be using a no-code trading platform.

Where is your competitive advantage?

Hedge funds exist because: well they simply are “better than the average investor”, or at least they advertise themselves like that. Okay, now assuming that they are “better”, in the sense that they do beat the market, what makes a hedge fund so special? After a lot of tweaking, analyzing papers, and more, it really boils down to a simple equation:

Great Data + Great Algorithm + Great Market = Great Hedge Fund

There you go, the secret formula to building a hedge fund. And every hedge fund out there has a competitive advantage in at least one of the three areas (the best bet is to have at least 2 out of the three). Now with this in mind: Great Data, Great Algorithm, and Great Market, can we truly create market beating returns using a no-code trading platform where data and systems are limited to tickers and candles from exchanges? Can we truly create market beating returns if we’re not leveraging asset derivatives like options to reduce risk? Can we truly create market beating returns by not using alternative data like news or analytics to make better risk decisions? Just with these three facts in mind, we have simply covered all of the items that make a great trading strategy, there is simply not enough customizability to build a market beating strategic return. With no-code trading platforms, you are simply restricted to using technical indicators, and stop-losses to hedge your bets and risks, let alone complex strategies like complex portfolio rebalancing, smart order routing, and more.

There is simply not enough customization within no-code based platforms to allow for any part of the hedge fund equation to be improved upon, making no-code platforms not suitable for building fully automated strategies or super useful signals.

Abstraction and Control

If you’re an investor, this section might not be the best for you, but for developers and hedge funds, pay attention. Everything about investing is about control. Whether it’s market connections, server uptime, and API key management, ensuring that your strategies are up and running and secure are the utmost important. With no-code trading platforms, you are entrusting all of your code and technology to a web platform with limited customization and minimal exit strategies. Underlying functions and calls are done completely in-house and proprietary, with limited knowledge of implementation. What they gain in allowing for easy “setup”, they lose in customizability, control, and abstraction of your trading algorithm. What if the platform shuts down? What if the next day the servers are changed? These items are extreme considerations for anyone moving tens of thousands of dollars automatically via a platform.

Extensibility and Iteration

No-code is great if you have a strategy that specifically uses technical indicators. It’s great for plugging in a SMA here, an RSI there, and using those to make some sort of large conditional statement to make a decision. Cool, but what if we abstracted this to one more level, say that I wanted to first look at SMA and if it passes the “golden cross” test, then I want to look at RSI, but if it doesn’t, I want to look at support and resistance levels, just this conditional statement results in an inability to use a no-code trading platform. What if you built your own trading signal? You could not easily plug this into a no-code trading platform nor create the APIs and services to pass the results in.

How about actually conceiving this strategy? How do we backtest and test multiple configurations with varying levels of logic? With no-code trading platforms, you are restricted to specific parameters on specific indicators with limiting results (remember our equation, what about experimenting with other items like tickers or data sources). All of these make your strategies less effective and more risky compared to a thoroughly tested strategy.

What’s No-Code Great For

If you’re an investor or a trader who looks purely at technical indicators or want to automate your portfolio rebalancing and buy and hold strategies, no-code platforms might be super great for you as you’ll be able to easily iterate and improve upon your algorithms without knowing how to program. If you’re looking at a single source of data via price, then no-code is right up your alleyway since many platforms like Tuned, Mudrex, and Trality integrate well with many crypto integrations. Composer takes an even more interesting approach to no-code strategies with its fun-to-use building-blocks like design.

What’s the alternative?

Now if you’re serious about building trading algorithms and generating algorithms just like the pros do, you’ll have to look elsewhere. As we mentioned, the other spectrum is the DIYBIY mentality and that’s what many hedge funds focus on doing (well at least the top 4%), with millions of dollars to spend on infrastructure, these hedge funds attempt to build everything out in-house. And yet this spectrum is also dangerous as you enter the side of being too bogged down on infrastructure and too little time to focus on building a strong algorithm. Fortunately, there is a middle. Trading frameworks out there enable users to build trading algorithms at record speed without infrastructure headache, in addition, the transition to actually implementing strategies are almost painless (and sometimes even easier than no-code). For example, take a look at this tutorial below where we used the Blankly Package to build a bot in 25 lines of code (literally 6 minutes). Without the worry of handling infrastructure and the ease of transitioning to programming while also maintaining the ability to rapidly build, and expand your trading algorithms in data, algorithms, and markets, trading platforms like Blankly are your best bet if you want to build meaningful trading algorithms that truly beat the market.

If you’re looking for the best ones, I definitely encourage you to take a look at this comparison.

Conclusion (The TL;DR)

Algo-trading is growing and it’s a new asset class that unlocks a whole flurry of potential to beat the new market
Don’t get stuck on using no-code trading platforms that lack customizability, abstract control and risk, and aren’t extensible as you will be left in the dust by the many adopting more complex strategies based on the hedge fund equation
Instead use an easy to use trading framework such as Blankly to rapidly build and deploy trading models at scale without sacrificing the ability to build complex strategies and data sources
Keep building and keep trying 😄

An Honest Comparison of VS Code vs JetBrains - 5 Points

Jeremy Liu — Sun, 09 Jan 2022 22:01:06 +0000

Call me crazy if you have to. You’d think that after five years of coding in one IDE, I would be out of my mind to leave the VS Code. And sure, before I tried out the JetBrains ecosystem, I would’ve completely agreed. I would’ve even offered up my left kidney, fighting for the fact that VS Code was superior (just like M1 Pro Macs are) to any other IDE on the market. Now just a bit of background, I work at Blankly where we enable people to build trading algorithms on hedge-fund level cloud infrastructure in minutes instead of months. During one of our daily stand-up meetings, my co-worker, Emerson, was adamant about the JetBrains ecosystem and extended the meeting trying to convince one of us to give it a whirl. And… I begrudgingly agreed, thinking it’d be a one and done thing (and so that the finally meeting could end). But wouldn’t you know it, I’m here now, writing this article about what finally convinced me to abandon the IDE that has been by my side since day one.

If you’re looking for a change of an editor, but are hesitant on even giving JetBrains a try (because, I mean VS Code is free and JetBrains for non-students isn’t) or if you’re just interested in the reasons as to why I committed such a betrayal, read on! This article is an honest comparison between VS Code and JetBrains and why the switch to JetBrains might be great for some.

1. Code Inspection & Refactoring

VS Code - Quick, Easy, Multilanguage Support

VS Code can be considered as an “editor” for a reason. It’s quick and easy to get up and running with just about any programming language. And, for any full stack developers like me, this is big. Whether you’re switching between python for an API or javascript for a frontend, or adding a NextJS react app, or setting up a ruby on rails system, VS Code can support these languages, provide linting, and much more—entirely out of the box. And, on the off-chance it cant? Well, simply just find an extension!

Also, VS Code has great linting thanks to its many new extensions such as Github Copilot, AI-based linting, auto imports, and much more. It’s now easier than ever to get what you want, when you want it. All you have to do is add a period and most of the time, things just pop up. But sometimes, it gets super frustrating when it suddenly doesn’t. In fact, more often than not, I’ve been bogged down, trying to figure out why a specific linter would NOT work. Whether it’s because of my multiple python environments installed via Anaconda or missing package that aren’t installed—most of the time—I simply have no idea. Moreover, linting JavaScript is also kind of a doozy. VS Code just doesn’t even try to infer Javascript type. But, luckily for me (and if you’re developing in an enterprise context too), TypeScript solves these issues, making this concern not really come up.

Here we have broken linting as our packages are not being recognized as existing even with a pip install. Turns out it was because we forgot to change our python environment.

Finally, let’s talk about refactoring. I think VS Code does a phenomenal job of refactoring as an editor—key word being editor here. Being able to refactor a variable name change or moving a file and refactoring imports is great. But, say that I want to say move a function, change a set of named parameters, or abstract code out, VS Code is limited in its abilities. Luckily, the base refactoring does satisfy many use cases, and it fulfilled most of my use cases as a student for the past five years, albeit minimally at times.

JetBrains - Specialized, Specific, and Powerful Support

For JetBrains, man is this a powerful IDE with unreasonable amounts of settings. My first time opening this IDE, I spent over an hour messing around with my settings, getting my code to display just right. But, what took a bit of time to get use to was having to switch between different IDEs for different use cases. If suddenly my POST request stopped working, I would have to open up PyCharm to see if the problem was on my backend or, if I suddenly thought of a better optimization for my OPTTSP traveling salesmen class project, I would have to load CLion as well. But, with Intellesense, opening the different IDEs was simply learning a couple other scrips like webstorm . and pycharm . instead of code .

Secondly, the capability of the JetBrains engine was surprisingly noticeable once I had switched over. Gone were the days where linting would randomly appear and I would try spamming command+p -> reload windows hoping some of the OCD triggering red lines would go away or actually tell me something useful. Instead, simply having consistent and fast-responding linting has been a complete breath of fresh air.

One keyboard shortcut to see every use

Finally, what really stood out to me in JetBrains was its refactoring refactoring capabilities. Just last week, I was working on finalizing the private beta platform for release. In the process, I reorganized and generated new components to make it more scalable for future development. I think in total I moved and broke apart over 200 components and not once did I run into a single compile error caused by important statements or invalid/undefined components. For contrast, just reorganizing two files in one of my projects for my data structures class broke my entire cpp code, requiring me to manually change some imports and functions. Moreover, JetBrains extensive refactoring tools like safe delete, global renames, and more has ensured that I have had more tools than I could ever use.

Now easily view context of all usages and be confident about refactoring & renaming

JetBrains IDEs read the headers for you

Comparison

Overall, I would say that both of these environments come close to in terms of features and functionality. Both have solid auto-linting and formatting functionality and add a variety of customizable colors and swiggles to help you debug and better visualize your code. However, with JetBrains perfectly consistent linting engine coupled with its unbreakable refactoring process, I must say that if code factoring and refactoring is important to you and your workflow, I would definitely recommend JetBrains.

2. Debugging

VS Code - Debug Almost Anything

The beauty about VS Code debugging is that it’s super extensible. Every time you hit the run button on the left bar, VS code generates a .vscode directory that houses your settings.json that houses your debugging properties. For most languages, such as debugging python or JavaScript, this is really easy since VS Code handles the debugging all for you, and assuming you have your environment set up correctly, debugging is as easy as clicking that button. It’s also super easy to change what you’re debugging by simply going to the settings.json file. Now it gets more complicated when you’re using build specific or even platform specific languages such as C++ or C where setting up gcc and clang ups the complexity. Setting up the ability to debug these files becomes such an annoyance and and takes endless amounts of time. I’ve spent many a days just copying over previous settings.json to try to get my current project to work. At my uni—The University of Michigan, Ann Arbor—they’ve defaulted to simply give everyone the same settings.json to use in order to hours of office hour frustrations. But, the fact that we even need to touch this settings.json to handle debugging is in itself a little bit frustrating.

A simple settings.json needed to run the debugger C/C++ on MacOS

Now as for the actual debugging, placing breakpoints, identifying variables, adding watchers, VS Code handles beautifully in their debug console. However, it would be nice to see the variable values directly on the code overlay rather than in the side panel.

Luckily, what’s beautiful about VS Code is that the extensions, and support for a wide variety of languages enables everyone to set up debugging in typically seconds, and if not minutes. The debugging does a great job for minor debugging and smaller cases but when it comes to specific languages, VS Code debugging breaks down. However, I’ve also tended to notice that require larger heap sizes (i.e. recursion or simply large function calls), the debugger seems to struggle and crash further into its runtime.

JetBrains - A Debugging Monster

On the other hand, because all of Jetbrain’s IDEs are built on configuration based runs, you can begin a debug session at the push of the debug button. Breakpoints can be set globally in the IDE just by pressing the blank space by the line number. This instant setup for any debugging process makes the setup experience alone amazing. The IDE also shines during actual debugging. When in a debugging session, all variables defined in the scope are visible by their definitions. This allows an extremely easy way to see the values just by clicking. I was impressed a few days ago when I was running a debug in Pycharm and attempting to view the values of a dataframe. Just by clicking the dataframe variable and pressing view as dataframe, Pycharm opened the Dataframe in SciView and showed all Dataframe values as well as the column headers:

Insight achieved by setting a breakpoint natively and hitting debug

In the screenshot above, all local values in the scope are viewable in the bottom window. The dropdown next to history_and_returns shows all attribute values of the dictionary and the dataframe nested in that dictionary. On the right, Pycharm is showing the same Dataframe that is already nested in a dictionary as a SciView. Achieving this level of insight into your code without setting up any print statements or stack traces is extraordinary useful. You imagine how easy it is to find flawed logic in a loop, fix indexing errors, or any other traditionally abstract reasoning when all values are projected right into your editor next to their assignments.

Like many other debuggers, JetBrains also offers the ability to step through your code. This includes the classic step over which goes to the next line or step into which goes into any function you want to examine further. An extremely useful one is Run to Cursor which allows you to set what feels like another breakpoint just by placing your cursor on a line you want to examine further. The ability for the JetBrains debugger to be instantly setup and then provide such an integrated experience has completely changed how I code and accelerated my development.

Comparison

As debugging is one of the most common (and rage-inducing) things we do as developers on a day to day basis, I must say this is an important, if not essential feature for many developers to have. Both IDE environments provide a very solid debugging environment but, I must say that JetBrains edges out VS Code again here by just a little bit. Specifically, with JetBrains displaying variable data right next to the declared variable makes tracking the values inside variables much more manageable where there are numerous variables live. Moreover, JetBrains stronger and more stable debugger that does not require complex set-up like the settings.json (which also, at times led me to use cout or printf statements instead) in VS Code was the cherry on top. These factors just come together help save me that tiny bit of extra time in my day and makes the JetBrains more attractive to me.

3. Git Integration

VS Code - A Powerful Source Control Built In

Anyone that’s working in teams or even cares about storing their code somewhere safe (in case you throw your computer out the window when your code isn’t running) knows how important git is in their workflow. Git version control is almost imperative now for any modern code editor. And VS Code does this well. VS Code will automatically detect an initial git repository and immediately offer many inherent git commands available including push, pull, commit, etc.

From VS Code’s git panel, I’m able to easily sync changes that I have and also see what changes have been made. I’m also able to create branches and clone repos (though some of these are hidden in the more). What I love the most about VS Code is that it sort of tells you what to do. It immediately tells you to commit changes and detects all files that have changed and allows you to commit a message. It also checks and syncs changes as you go. It detects local branches vs remote branches, and has solid rebasing capabilities.

Easily see where the merge conflict is inline

One of the things that makes VS Code stand out too is the ability to handle merge conflicts. With merge conflict resolution built into the VS Code editor, I’m able to press a button to keep a current change, or handle the incoming changes that come in. And trust me, this has saved me so, so much time.

JetBrains - Never Touch the Command Line Again

I have barely touched my terminal in the entire time since I’ve switched over. With JetBrains providing full integration of merging pull requests, resolving conflicts, and switching and comparing branches, the source control is unreasonably better than my experience in VS Code. Take a look at the experience doing some of these common actions:

Comparison of a particular file between branches

Branch details built in

Details git log with deltas

Comparison

In terms of features for Git integration, these environments are basically identical in terms of features. I found that they provide basically the same functionalities and you could not go wrong with either one. I would say that this just come down to personal preference. For example, I found myself preferring the JetBrains method with how merge conflicts are resolved as the interfaces laid side to side instead of stacked on top of each other. Not really a determining factor in the end.

4. Extensibility

VS Code - Extensions Galore

Visual Studio Code is one of the most extensible editors out there. Integrations and extensions are at the heart of the VS Code editor. Top picks include the Python Extension, Remote Development Extension, and many other IntelliSense driven extensions. VS Code also has some cool ones including code formatting via Prettier, theming via icons and code editor themes. Just about every single item / feature that VS Code offers is fully extensible and there probably many of extension out there that can empower your process.

One of my all-time favorites is the remote docker container support. This allows users to use VS Code to remotely program in a docker container itself. If you have docker installed locally or remotely, you’re able to easily run your code and do all that you need to do in docker, simply from your vscode. Want some more interesting stuff? Check out remote development via SSH. Microsoft allows you to remotely SSH into your server’s development environment and program just like you would normally. All of these features here, makes VS Code arguably one of the best editors out there, as these are simply one click away from being fully operational.

JetBrains - A Deep Ecosystem of Integrations

Extensibility is an area where a JetBrains IDE doesn’t necessarily shine. This is largely because you will find that everything that you need often ships with the IDE. With the benefit of being able to install a specific IDE with superpowers for your language, I’ve found that the integrations that I might be used to installing on VS Code shipped with the software.

For example, JetBrains powerful built-in integrations with docker. By only specifying a configuration type such as a Dockerfile, all JetBrains IDEs give complete control of all arguments, names, tags, ports, environment variables in an easy-to-use configuration GUI. When run, the IDE integrates with docker integration to bring you the build log, the run log, environment variables, and easy-to-read and integrated configuration settings:

The same is true for many other common services such as FastAPI, Flask, shell scripts and a ton more.

JetBrains IDEs also have a rich plugin ecosystem. For example, I was able to install a plugin that offered full language support for Verilog and another that gave full support for Matlab. Ironically these lightweight JetBrains plugins gave a better coding experience than the native Matlab or Quartus (Verilog) environments.

Comparison

Indubitably, both have an extensive community and marketplace to be able to get a variety of stable and custom built extensions/plugins. Any feature that each respective IDE might not be able to handle, you will most likely be able to add it to their core functionality. However, with a slightly larger community base on VS Code (and the support of Microsoft), having more extensions and the life-changing Remote-Containers extension (for me, at least) allowed me to iterate so much faster. Here, if you are working with very custom code that might require custom extensions like Docker, VS Code is definitely the IDE for you.

5. Collaboration

VS Code - Live Share by Extension

Though VS Code itself does not have live sharing built in. Microsoft has created a live share extension that allows almost instant live sharing. What makes this so great is that all anyone needs is VS Code, in fact now you can even live share directly from your browser using vscode.dev. The actual live share process itself is really good...as long as you have a good internet connection. Users can follow each other and edit code alongside each other. VS Code also keeps track of who helped author the commit for source control. With all of this in mind, and how simple live share is to set up. VS Code arguably outshines almost any other IDE and editor in this space.

There are some caveats. One example is live sharing Vue.js. When live sharing vue, some of the extensions don’t propagate including Vetur, which is critical to Vue visualization. This sometimes makes it extremely difficult and arguably annoying, but I think this is very specific to a specific audience (so Vue users out there beware). Also, I really hate how the undo functionality is tied to the machine, not the user. I just gets in the way and causes confusion.

JetBrains - Secure & Distributed

All JetBrains IDEs offer a huge number of options to share your code with others and collaborate live. These options vary by the degree of security they offer. An impressive ability I recently discovered was running any JetBrains IDE inside a docker container using projector. This allows me to connect to a JetBrains IDE hosted on a server such as the cloud and then code with full JetBrains features inside my web browser. Now with just a password I can securely code anywhere from a headless server. This is just one of the many sharing options.

A more mainstream way to share JetBrains IDEs is using Code With Me. This allows you to view other people’s projects directly in your IDE and and use their dev environment as if it was native on your computer. One thing I was impressed with is that a teammate was having a python issue and easily started a Code With Me Session. I was able to natively run their configurations, use the debugger with the same insights I showed above and and then easily fix their issue.

The many different and well integrated ways to share IDEs are amazing for people trying to optimize their security, collaboration or how they work with distributed teams.

Comparison

If it was two years ago, I would've deemed this feature pretty irrelevant. In fact, before two years ago, I never even know collaboration features existed in IDEs. We could've simply walked over to our colleague(s) and started working together on the same machine. But now, with COVID still going strong, it has become much harder to be able to have this luxury. Because of this, both of these IDEs do a very solid job, enabling such functionality. However, the sole reason that the undo function i synced across users instantly make me recommend JetBrains a hundred times over VS Code. And, the video and audio call support alongside the ability to record Git blame across users is just the frosting on top.

Now, beyond these factors here, I also know that VS Code is a free IDE while JetBrains comes with a decently hefty price tag which I understand might turn some of you away (it is free for students though). However, for me, its been a blast joining a part of the JetBrains ecosystem for this past month and I cannot wait to continue working with it more. I hope you might consider giving it a shot, even when there is a bit of a price to entry.

A converted full-stack developer,

Jeremy, Lead Engineer at Blankly

Trading Algos - 5 Key Backtesting Metrics and How to Implement Them in Python

Brandon Fan — Sat, 08 Jan 2022 20:41:28 +0000

Metrics surround us. Whether you're building the next big thing and need to measure customer churn, retention rates, etc. you will always need numbers to really back it up. This is the same when we are building trading strategies. We need easy and straightforward ways of analyzing whether or not our model is actually doing well. Now before we go into understanding various portfolio metrics, we have to first really understand the point of having good metrics. And for that, we have to understand what it means for a model to "do well".

For some of you readers out there, you probably already know what you want out of a strategy and what metrics you may want to do. For that, feel free to skip ahead to the various metrics and their implementations.

What is "Doing Well"

Now when you think of a good investment strategy, what do you think the most important thing is? First and foremost, at least for most people it is, "I want to make the most amount of money possible", i.e. your return on investment (which can come in many forms too as we will soon see). But then this begs the question of a multitude of things: "when do I get my returns?", "how do I get my returns?", "do I care about my returns every year", and, arguably one of the MOST important questions, "what are my chances of NOT getting my returns", i.e. my risk. Ahh, you see now the question gets much more nuanced as there are a lot more factors. So I propose a different statement of doing well. Doing well is not just about how much money you make, but also about when you make it and how much risk you are taking on.

A model does well when it successfully produces high returns year-over-year while reducing the amount of risk for those returns

That's awesome, now how do we actually measure that? How do we measure risk, returns, and all of these other factors? Well that's why we have metrics.

Introducing Strategy Metrics

Now for many people new to quant or even pros probably don't know all of these metrics and each of these metrics might be used for a variety of use cases. Before we go into them, I think it's super important to keep in mind that not one metric tells an entire story, just like a person is not just governed by their name. No one metric can tell you whether or not a stock is a good buy or not, nor can one metric tell you if your model is good or bad.

So let's get into it. We're going to go through 5 useful metrics in this post. If you want to see more metrics or don't want to implement all of these yourself, you can take a look at the Blankly Package that is one of the fastest and easiest to set up trading packages out there. Also, here are some great explanations on some of the metrics that we go over today.

I've implemented a gist that houses all of the metrics you'll see below 😉.

Compound Annual Growth Rate - Starting Easy 😉

Compound Annual Growth Rate (or CAGR for short) is one of the most important metrics and the most traditional metrics out there. What does the CAGR tell us? Well it takes our cumulative return (how much did we cumulatively make across X years of a backtest or a strategy), and finds the percentage that makes it such that your cumultaive return was equal to your portfolio growing at Y% per year.

Okay, that's a little confusing, so let's use an example. Let's say that your strategy started out with $100 and after 5 years you turned your money into $500. Well what's your cumulative return? (500 - 100) / 100 = 400%

But what does that tell you about how much you made over the years—i.e. what is the "annual return" of your portfolio? With that, we can then easily compare this to things like the SPY (which has a CAGR of 9.9%)? To do this we annualize our return with the following formula:

Shoutout Investopedia for a nice and beautiful formula :D

Now using this formula we can see that our strategy has an annualized return of (500 / 100)^(1/5) - 1 = 37.97%.

Nice! Now how do we implement this formula? As you can see it's pretty straightforward, if we have our start and end balance.

account_values = [100, 200, 300, 400, 500]

def cagr(start_value: float, end_value: float, years: int):
    return (end_value / start_value) ** (1.0 / years) - 1

cagr(account_values[0], account_values[-1], 5)
# 0.3797...

Great! Super easy and straightforward.

When would I use this?

Use CAGR as a way to easily compare your returns across strategies. It's much easier to analyze annualized returns because various strategies will have been run longer than others. When you're backtesting, you might run one strategy for 30 years whereas another one for 10. These will clearly have different cumulative returns so you will HAVE to use CAGR.

What is a Good CAGR?

As mentioned before, we use CAGRs to compare with different benchmarks such as the SPY, BTC-USD, Russel 100, etc. thus we can now compare our CAGR with these benchmarks and figure out whether or not our model is beating a traditional buy and hold. You should also compare this to what you want to make (for example real estate returns on average 10-12%, current crypto defi rates such as at Celsius provide upwards of 8.50% in CAGR)

Sharpe Ratio - Risk/Reward Ratio #1

The Sharpe ratio is one of the most renowned ratios out there because it captures two things: 1. The amount of reward you are getting, i.e. your returns, but also 2. how much risk you are taking on. It then takes this and makes it into a metric that can be verbalized as "for every unit of reward, how much risk am I taking on?" or put another way, "what is my reward to risk ratio", or another way, "Am I being rewarded for my risk?", now with that in mind, I think most of you guys can guess that it's going to include some sort of division. And indeed, it does! But the question we have to ask ourselves is...well how do we measure reward and risk? The ratios use something very different from CAGR. Instead of using account values (your total asset value at specific times), it uses returns or deltas (changes in your account value at some times), and uses these to ultimately calculate risk (the standard deviation in returns) and reward (the average in returns).

So why do we use standard deviation of returns?

Standard deviation, if you remember from stats class is a measure of how spread out your data is. In the case of returns, how spread out your returns is, means how volatile your account value is. The more spread out your returns are, the more likely you are to run into losses, and therefore more emotion and therefore more volatility. I mean who wants to see their strategy go from up by 7% on one day, and then drop by 50% the next day 💀💀!

What is Risk Free Rate?

Our Risk Free Rate is our benchmark for how much we would make if we chose a "risk free investment," as typically this is the treasury yield, or can also be the estimated percentage that you would make if you did a buy and hold, or whatever you define as "risk free".

Now what the heck is that square root of n?

Now we have a list of returns, but have no concept of time, how do we know if the returns are on a per hour return or on a per day? As you can see, the various sharpe ratios are going to be different depending on the sample time and sample size (don't forget about your stats class and standard error, it's ok I kinda also forgot).

Don't believe me? Think about it, do stocks move more in a minute, or do they move more in a day? Every minute, a stock might move 0.01% but every day, stocks might move 1 to 2%. Well, clearly the sharpe ratios are going to be very very different. That's why we have to annualize them all so that they are consistent and easily comparable. This makes it possible for us to compare strategies that are running on minute data, and strategies that run every hour or every day (it really doesn't matter).

Implementation (PAY ATTENTION)

Now to implement this one, we'll have to do some manipulation to our account values. Let's use the power of numpy to help us out here (oh and it's also the same in pandas too. We'll be using np.diff to take the returns of our account values and resampling them.

import numpy as np
from math import sqrt

# our account values every month
account_values = np.array([100, 200, 400, 300, 250, 600, 575, 425, 325, 800, 900, 1020])

def sharpe(account_values: np.array, risk_free_rate, annualize_coefficient):
  diff = np.diff(account_values, 1) / account_values[1:] # this gets our pct_return in the array
  # we'll get the mean and calculate everything
  # we multply the mean of returns by the annualized coefficient and divide by annualized std
  annualized_std = diff.std() * sqrt(annualize_coefficient)
    return (diff.mean() * annualize_coefficient - risk_free_rate) / annualized_std

sharpe(account_values, 0.099, 12)
# 0.9292540351869186

Now hold up...🤔

For the astute observer, you might have saw that the sharpe ratio formula doesn't have an annualize coefficient in the numerator but the actual formual does. And we also only annualize the denominator. Why is that? That's because the statistical formula assumes the returns are in the unit of $X and not as a ratio (i.e. $ / time) This is why in practice, we multiply in the numerator and only in the denominator as well. This will then give us our correct result 😃.

What's a Good Sharpe Ratio?

As we mentioned, the sharpe ratio is a ratio of reward vs risk, so anything greater than one is phenomenal. This means that for every unit of risk, you are getting more reward. A shapre ratio of 2 is phenomenal, a sharpe ratio of even higher...well you might be onto something amazing (or you might need to check your calculations)

Great! Now let's talk about a similar ratio. The Sortino Ratio.

Sortino Ratio

The sortino ratio is almost the exact same as the sharpe ratio, except one difference: instead of analyzing ALL the standard deviations of returns, we only care about the negative returns. Why do we do this? Well I don't care if my model necessarily makes 25% and then 2% the next, they're all wins in my book, what I care more about is that I don't want to be losing 25% one day and then 2% the next day, I'd much rather only lose 2% maximum.

Implementation

The implementation is almost the exact same, but we'll use some more beautiful numpy magic here to help us out. We'll use boolean masking (it's actually amazing)

import numpy as np
from math import sqrt

# our account values every month
account_values = np.array([100, 200, 400, 300, 250, 600, 575, 425, 325, 800, 900, 1020])

def sortino(account_values: np.array, risk_free_rate, annualize_coefficient):
  diff = np.diff(account_values, 1) / account_values[1:] # this gets our pct_return in the array
  # we'll get the mean and calculate everything
  # remember, we're only using the negative returns
  neg_returns = diff[diff < 0]
  annualized_std = neg_returns.std() * sqrt(annualize_coefficient)
    return (diff.mean() * annualize_coefficient - risk_free_rate) / annualized_std

sortino(account_values, 0.099, 12)
# 2.954078445173037

When would I use Sharpe vs Sortino?

Again, going back to what I said early, no one ratio or metric tells the whole story. Sharpe and Sortino are great for varying things. Sharpe will tell you your overall standard deviation, and Sortino will tell you your reward relative to how volatile your downside risk is (which induces the most emotion). If you care about overall reduced volatility, use sharpe, if you care about making sure that your model reducing negative downside, then use sortino.

Maximum Drawdown

Okay, let's stop looking at ratios, and focus on another metric. Maximum Drawdown. The maximum drawdown is as it sounds: it's the largest drop between a peak and a trough. It shows the largest swing you're expected to have during a trade or strategy and is actually used to calculate the calmar ratio (here's the implementation). This is important, well because we want to know how long our model is going to "lose money for", or just simply what the biggest loss our strategy incurred over a period of time. We can also compare this to our benchmarks and see if we're losing more money and introducing more volatilty than our benchmarks.

Implementation

For this implementation, let's use pandas because we can use some pandas magic and their functions to calculate the cumulative product of the returns and subsequently the associated peaks. How does pandas do this? It takes the cumulative product at various points and then finds the maximum peak as it goes (making that a series), it then takes that and finds the lowest values by using cumulative and finding the ultimate drawdown.

account_values = pd.Series([100, 200, 400, 300, 250, 600, 575, 425, 325, 800, 900, 1020])

def max_drawdown(account_values):
  returns = account_values.diff(1) / account_values[1:]
  cumulative = (returns + 1).cumprod()
  peak = cumulative.expanding(min_periods=1).max()
  dd = (cumulative / peak) - 1
  return dd.min()

max_drawdown(account_values)
# -0.6381664371434193

What is a good drawdown?

Drawdowns are always negative...so naturally we want less of that negative number (so the closer it is to 0, the better). Unfortunately, we can't ever hope to have a fully 0% drawdown, so the closer it is the better, typically seeing something that's less than 10% in drawdown is amazing.

Value-At-Risk

Now, Value-at-Risk is a complicated metric as it's not super super intuitive to visualize as there isn't a direct formula. Value-at-risk (VaR) is very straightforward: what is the maximum capital am I risking at any given point in time in the portfolio (i.e. the average value at risk at any given point in time). Think of it as like a sample of the means. Thus, we use a confidence interval to determine how confidentally we can estimate the value at risk. To do this, we take your returns and create a normal distribution (wow the stats is all coming back 🤯) now that's great and all. But a faster way to do this is to sort all the returns (from negative to positive as they will naturally create a normal distribution), and take the associated confidence interval.

Implementation

import numpy as np
from math import sqrt

# our account values every month
account_values = np.array([100, 200, 400, 300, 250, 600, 575, 425, 325, 800, 900, 1020])

def value_at_risk(account_values, alpha: float):
  initial_value = account_values[0]
  returns = np.diff(account_values, 1) / account_values[1:]
  returns_sorted = np.sort(returns) # sort our returns (should be estimated normal)
  index = int(alpha * len(returns_sorted))
  return initial_value * abs(returns_sorted[index])

value_at_risk(account_values, 0.95)
# $59.375

Interpreting the Results

What's beautiful about VaR and Conditional VaR is that they are outputted in the unit of the asset, so we can directly translate that into something we can directly visualize and see. The more value that is at risk, the more we should be worried about each trade that is made. Thus, we want to minimize this as much as possible. I mean...I want to sleep at night too.

Beta

The final metric we're going to look at is Beta. Now most of you guys have probably heard of alpha (1. it's before in the greek alphabet, and 2. it's the amount of returns you have made in excess to the market), but beta is something that looks again at something that we care about: volatility or systematic risk. Specifically, beta compares your strategy to a designated benchmark or asset (say SPY, BTC-USD) and compares your strategies returns relative to theirs. The higher the beta, the more volatile your strategy is "relative" to the base asset, or the more systematic risk you are taking on, check out article by Investopedia. Thus, the closer the beta is to one, the more aligned it is with the base asset, and a beta less than one means that your strategy is less volatile (which can also mean you can sleep better by using your strategy INSTEAD of using buying the base asset, and everyone wants that).

Implementation

Now implementing this in practice is actually slightly harder, because we actually have to get market data. There are a multitude of different ways to do this. Some work in "research" environments and non-static data, others work in "liver" data, but we're going to use one that uses both: Blankly. Blankly is a package that enables the rapid development of trading algorithms at scale, extremely quickly (I built an RSI screener bot in 25 lines of code), and allows for direct connections with live exchanges for free live trading out of the box! Now there is some set up that you have to do, but fortunately all you have to do is run a simple command:

$ blankly init

And your entire directory is setup. You can check out more info here.

To actually implement this, we're going to make our function a lot more powerful by incorporating market data to directly calculate beta:

from blankly import Alpaca, CoinbasePro # supports stocks, crypto, and forex
a = Alpaca() # initialize the Alpaca Exchange

account_values = np.array([100, 200, 400, 300, 250, 600, 575, 425, 325, 800, 900, 1020])

def beta(returns, baseline='SPY', return_resolution='1m'):
  # get the last X month bars of the baseline asset
  market_returns = a.interface.history(baseline, len(returns), resolution=return_resolution)
  m = np.matrix([returns, market_base_returns])
  return np.cov(m)[0][1] / np.std(market_base_returns)

Great, now we're hooked up with live data in real-time. And the beauty of it is that we can easily switch this to use crypto by changing our exchange and that's all we need to do! A full function that works across assets.

Don't want to use blankly for stocks? We can also use yfinance, and the code would look like this (with some restrictions):

import yfinance as yf

account_values = np.array([100, 200, 400, 300, 250, 600, 575, 425, 325, 800, 900, 1020])

def beta(returns, baseline='SPY', return_resolution='1m'):
  # we have to restrict this to get the past year
  ticker = yf.Ticker(baseline)
  market_returns = ticker.history(baseline, period='1y', interval='1m')
  m = np.matrix([returns, market_base_returns])
  return np.cov(m)[0][1] / np.std(market_base_returns)

Why is Beta important?

Beta is super useful for us because we want to know if our model is gaining a higher reward while also beating the base asset or benchmark (i.e. less volatile and a reduction in systematic risk compared to the overall market). This is beautifully paired up with all of our metrics that we've seen earlier and allows us to get a complete and holistic view of our trading models.

Let's Sum It Up

The Importance of Testing

Okay wow... that was a lot. But I hope that this tutorial was super useful for you to easily see how we can use metrics to make better decisions about our strategies. Again, make sure you are testing all of your models before deploying them live or putting real money in. But also, don't ever forget that backtesting can only tell you only a certain amount: past performance never defines future success 😃.

In this article we learned about 5 metrics:

Sharpe Ratio - Reward / Risk Ratio across all returns
Sortino Ratio - Reward / Risk Ratio across negative returns
Maximum Drawdown - The largest loss during a backtest or strategy period
Value-at-Risk - The value at risk that is made per trade
Beta - The volatility of your strategy relative to some benchmark or base asset

Always Use Metrics as Comparisons and Not in Isolation

Remember that we don't use metrics in isolation to validate our model. We have to compare it to a variety of other models, strategies, and benchmarks.

How do I easily improve my strategies and these metrics?

It's hard to say how to best improve every strategy because each strategy is fundamentally different (for example strategies that trade on specific assets are going to be limited by the volatility of the underlying asset). But I have a couple of recommendations:

Use Stop Losses

Stop losses are great ways to make sure that you are not losing money. Investor's Business Daily has a great article on this that you can find here on cutting your losses short. This will greatly improve your Sortino ratio and sharpe ratios if you're able to adequately exit out of losing trades. You are as good as your losses are 😃.

Make More Frequent Trades

This one might work both well or poorly, but the more trades you make the more "small wins" you will have which will naturally reduce your volatility. This might not change your sharpe ratios and sortino ratios, but depending on your interval, various trades will result in better results. This combined with stop losses will also really help.

Iterate, Iterate, Iterate

Nothing can beat iteration and rapid optimization. Try running things like grid experiments, batch optimizations, and parameter searches. Take a look at various packages like hyperopt or optuna as packages that might be able to help you here!

Signing Off Here!

That's all I have! I hope you thoroughly enjoyed this post, and hopefully this helps you along your quant journey. If you want all of the metrics, check out this gist

If you still have questions, feel free to reach out to me over twitter @bfan1256 or join this Discord where we talk about building strategies as much as we breathe ;).

If you’re excited about this space and quantitative finance, we love to hear more about why you’re interested and encourage you to get started by building your own trading model using Blankly.

Cheers to all the quants out there! Keep building 😊 🚀

How To Add A Community Trading Bot on Discord

Kevin Wang — Sun, 02 Jan 2022 21:10:01 +0000

At Blankly, we pride ourselves not just for the product that we are building, but also towards cultivating a community of quant enthusiasts which no one else is doing in this field right now. We identified a critical void that many Discord servers currently suffer from: low member engagement. That said, to elevate the overall experience of using the Blankly package and fix the problem identified earlier, we decided to roll out three Discord bots each of which specializes in something unique. We will dive into the Blankly Connect Bot in this article.

Using Blankly Connect to Integrate a Discord Bot

To give the users a taste of our product, we created this bot using a virtual sandbox and our in-house Blankly Connect API—the single source to connect, unify, and trade across all exchanges—which aims at simulating what the actual product will be like. We feel that this is the best way for beginners to get acclimated with the fast-paced environment of quant trading and for more experienced members to understand how our platform works.

Having used Discord previously as a means to connect with friends, group partners, and other communities, I was very excited to create a bot that focuses on community trading ultimately helping server owners spice their servers and elevate their member's experience to a new level. At Blankly, for example, it gave us the opportunity to differentiate ourselves from other similar platforms by offering our members something they haven't ever witnessed.

I would highly recommend that you install some kind of an IDE (I used VSCode, however my team prefers Webstorm :/), Node.js, and set up a Discord account. Before you even start coding, you are going to need to create an application from the Discord Developer Portal which will allow you to obtain an authorization token and set the permissions for your bot.

Once the application is created, click on the section titled Bot and go ahead and fill the necessary details. Find the bot's authorization token and make sure to not share it with anyone. This token will be needed in the next few steps. Now that your bot is setup, you have to invite it to your server. Therefore, select the OAUTH2 tab and then the URL Generator to allow you to select certain permissions and abilities for the bot. When done, copy the URL shown at the bottom of the page and paste it into your web browser. Select the server to which the bot will be in and that's it. Your bot is alive within your server... YAYY 🎆

Now, create a file on your IDE which will contain your bot's token like the following snippet below and make sure to give it a .json file type before adding it to .gitignore, so that the token wont be visible when you push your repository onto Github.

{
    "token" = "[token goes here]"
}

The second starter file will be the package.json so that it is easy for others to manage and install your package. Add the following fields:

{
    "name": "[Name of your bot]",
    "version": "[version id]",
    "description": "Enter a description",
    "main": "bot.js",
    "author": "Your Name Goes Here",
    "dependencies": { },
    "scripts": { }
}

The last file needed is where you will code how your bot should function and its behavioristics.

let Discord = require('discord.io');
let auth = require('./botAuth.json');
let blankly_connecter = require('../src/blankly_client')

blankly_connecter.setExchange('coinbase_pro');
blankly_connecter.setKeys({'API_KEY': '***', 'API_SECRET': '***',
'API_PASS':'***'});
blankly_connecter.setSandbox(true);

let bot = new Discord.Client({
    token: auth.token,
    autorun: true
});

bot.on('ready', function (evt) {
    console.log('{Status: Connect Bot is Connected}')
});

bot.on('error', function (error) {
    console.log(error)
});

bot.on('message', function (user, userID, channelID, message, evt) {
    if (message.substring(0, 1) == '!') {
        var args = message.substring(1).split(' ');
        var cmd = args[0];
        args = args.splice(1);
        switch(cmd) {
            case 'ping':
                bot.sendMessage({
                    to: channelID,
                    message: 'pong!'
                });
            break;
            // Add further cases if you need to..
        }
    }
});

This is the basic boilerplate code for a Discord bot which responds to commands. This code snippet is more than enough to help you understand how Discord bots operate and the syntax required to be able to operate them. Another great resource I would recommend are the documentation notes for the Discord bots such as Discord.io, which this bot uses, but you can also use Discord.js. It is also important to emphasize that there are many other ways that you can code your bot depending on your needs.

Furthermore, as you can see, I imported the Blankly Connect API in bot.js so that I can utilize its robust functionality. As shown from the diagram below, this API acts as a middleman between Discord and the major stock and cryptocurrency exchanges allowing for a smooth experience for the user when inputting commands.

You now have the basic tools needed to create your own community-based trading Discord bot. Have fun coding and I hope your bots don't take over the world!

Overall, this was a very interesting project as it exposed me to a new aspect of Discord which I was previously not aware of—the Developer side. Due to the pandemic, I have started using Discord almost every day and never really thought how easy it would be to create a bot to handle certain tasks. Although at first it may seem a bit daunting, it gets easier and easier as you become more comfortable with the developer environment. I can now confidently go ahead and create a bot within hours. Incorporating bots to your server will definitely increase the overall user engagement and will also significantly boost the number of users in the server and, thus, is a great addition to have.

As we look to continuously improve the bots, our next few steps are to transform this project into being open source, similar to our other products, so that the community can have a more direct involvement with our cause and we can make improvements related to your feedback.