Forex Millisecond Backtester

Source code available here: https://github.com/5ury44/GoBackTester

Motivation

Forex has been crowned the world's largest financial market with an average of 6.6 trillion dollars traded daily and worth of over 2.4 quadrillion dollars. Though our club focuses a lot on crypto and equities, I was inspired to create some infrastructure to trade on the forex market this winter break. The volatility and size of the market seemed interesting to me, and I aimed to develop a highfrequency backtester that could take advantage of miniscule changes in data by the millisecond to gain a profit. To my surprise, TrueFX.com had millisecond level data listed for the past year for a variety of exchanges. Using this as a starting point, I set off building.

Setup

The first issue I faced was deciding how to code the backtester. Python seemed like a go‑to option, but parsing hundreds of millions of data points while backtesting did not seem efficient on it. After further research, I ended up using Go, a language similar to C that could run blazing fast and take advantage of features like GoRoutines. A few libraries listed here made the switch from Python easier꞉ https꞉//github.com/goex‑top/awesome‑go‑quant

Gathering Data

One would think TrueFX would be amazing enough to organize thier links for easy download. Unfortunately this was not the case. Apart from logging in, the program had to search through a menu of years, months, and currencies to finally find the appropriate download link. After numerous attempts in Go, I decided to revert to Python for just this aspect, taking advantage of Selenium and BeautifulSoup. The code below logs in, finds the IDs of the appropriate months, and cycles through to download all the required currencies within that month.

Processing Data

To process the data, each CSV file had to be opened and read. Currencies could have more than one CSV file if they ranged over multiple months so the program had to combine all the months into one map per currency. The CSV file contained a bid, ask, and date/time. This was parsed and outputted to a map with the currency as the key and an "instant" struct as the value. Instants contained a time.Time, ask, and bid. The code is shown below.

Trading

Base Alpha

The two main classes required for trading are BaseAlpha and TradeEngine which both worked together to create and test an alpha. The baseAlpha struct is shown below and includes the necessary parameters to run the alpha:

The Alpha must provide a time to start and end trading. The current time.Time allows the alpha to access the exact point in time the backtester is at, in case this is required for the alpha to perform. The currencies []string is just a list of currencies in the portfolio that the alpha can trade between. Holdings is basically the same except contains a constantly updating map of the holdings of any currency. A tradeQueue will be empty by default but provides a way for the backtester to receive trade requests at any particular time from an alpha using the trade struct shown below:

To construct a trade, the alpha must provide the index of the exchange (ex. "eurusd" in the program's array of exchanges is 9). Inverted would mean we are going in the opposite direction of the conversion the CSV file provdied. If this is the case, we take 1/bid instead of the ask to approximate what that figure would be. Finally, we pass in the amount of currency 1 we want to convert to currency 2.

The BaseAlpha class contains an easy func to make a new trade using the two desired currencies and the volume. It also contains important funcs like initAlpha(), called once from the engine for any initialization code, and tradeOnTime(), called every ms for additions to tradeQueue.

Trade Engine

Whenever a new alpha is created, the initEngine() function is called. Essentially, this initializes the alpha, downloads the required data, and processes it into the previously mentioned map of instant arrays. Additionally, since the start time is not always at the start of a month, we binary search for the exact time to start that is closest to the given start time‑ positions map[string]int stores the current position in the instant array for each currency.

The main for loop in executeEngine() is where the real magic happens. This loop updates every 1 ms and starts with positions updating each index to follow the change in time. Next, tradeOnTime() is called so the alpha's trades can be initialized. The tradeQueue is then iterated through to execute all the trades for this millisecond, accessing data differently depending on inversion. Finally, we flush the queue for the next millisecond and add a point on the graph to output.

Graphing and Evaluation

Evaluation of worth was something that I struggled to understand with forex. My first strategy and the one I ended up implementing was just converting all the currencies to the first currency at any point in time to judge worth. However, this introduced a need to define what a "well‑performing" alpha even was. With this strategy, we assume that a good alpha will make the most money in terms of the first currency. However, if the definition is general profits, the evaluation would have to shift to analyze the worth of each individual trade because making the first currency Currency 2 vs Currency 1 could change profitability. For now, I have stuck with analyzing the first way.

For graphing, I turned to the go‑echarts library which allowed me to create a simple line graph over time. To make graphing less intensive, I allowed a resolution to be set in executeEngine() that could place a point every x milliseconds instead of having every single point on the graph.

Example Alpha

To prove that the tester could work and enable some real money to be made, I created a simple alpha that kind of followed mean reversion. Essentially, I banked on the idea that the trend of a longer term would hold with some short‑term fluctuations. I created a data structure called a meanQueue that would hold a max of 500 items and return a prediction for the next point based off these points as shown below:

In the alpha itself, we basically checked for when the point laid under the prediction to invest our money. Once the last 5 points were averaging a downward trend and we were above the prediction, our money would be taken out because this signals a peak. This simple strategy aimed to take advantage of some small fluctuations to make a lot of money.

The alpha itself was initialized and run as shown below in its own GoRoutine.

Result

On a test run from November 1 to November 3 on just EUR/USD, we acheived over 3.4x the original portfolio worth!

Takeaways and Improvements

I learned a lot from making this backtester especially through using Go, a previously unfamiliar language to me. I was fascinated that such a simple strategy could make so much money, and I hope this project allows others to build their own forex strategies. Overall, I think most of the improvements would be done through making testing faster. Although my tests usually ran in under a minute, I would like to make file downloads a lot faster and increase the use of GoRoutines throughout. Also, I would like to add more features that are in our equities backtester and possibly find a way to integrate Python alphas if possible. Previously, we discussed how performance of an alpha is subjective so more ways of evaluating (possibly trade by trade) could be beneficial.