Backtesting Investment Strategies with MATLAB - MATLAB
Video Player is loading.
Current Time 0:00
Duration 35:39
Loaded: 0.46%
Stream Type LIVE
Remaining Time 35:39
 
1x
  • descriptions off, selected
  • en (Main), selected
    Video length is 35:39

    Backtesting Investment Strategies with MATLAB

    Michael Robbins, Columbia University
    Malin Ortenblad, Columbia University
    Yao Shang, Columbia University

    Overview

    Learn how to define investment strategies and leverage the Backtesting framework to run backtests, analyze and compare results, and generate performance metrics for your strategies from historical or simulated data. 

    Highlights

    • Integration of other technologies seamlessly with the Backtesting framework
    • Integration of ESG factors into an investment strategy
    • Integration of GTAA factors into an investment strategy
    • Integration of Brinson attribution into an investment strategy
    • Complex path-dependent costs and fees and their effect on incentives and performance
    • Agent-based borrowing and lending and the effect on cash drag and forced liquidations
    • Significant modifications to the backtesting framework code
    • Backtest and compare multiple investment strategies
    • Visualize the results and leverage performance metrics to pick a strategy 

    About the Presenters

    Alex Roumi joined MathWorks in February 2020. His focus is on computational finance projects. Prior to MathWorks, Alex worked on the electrical design of airport buildings and airfield lighting of runways taxiways and aprons, including Dubai airport.

    Michael Robbins has been the Chief Investment Officer of five large investment firms, including a bank with 8½ million clients. He has managed pensions, endowments, family offices and was the Chief Risk Officer for the State of Utah’s systems. He is a professor at Columbia University where he teaches quantitative investing, including graduate classes in Global Macroeconomic Tactical Asset Allocation (GTAA) and Environmental, Social, and Governance (ESG) investing. Michael specializes in governance, asset allocation, and manager search and selection. Look for Michael’s new book, Quantitative Asset Management: Factor Investing and Machine Learning for Institutional Investing, in January of 2022. Published by McGraw-Hill.

    Malin Ortenblad led a team of graduate students that used machine learning to study a European hedge fund in Michael’s Environmental, Social, and Governance (ESG) course at Columbia University. She then helped manage dozens of students for Michael’s Global Tactical Asset Allocation (GTAA) course as a CA. In December of 2020, she graduated from Columbia University with a Master’s in Business Analytics. Malin has expertise in healthcare strategy consulting including machine learning during a cancer research project for Frederick National Laboratories, a hospital in the National Institutes of Health and Cancer Research Institutes systems. She is now working as a consultant in healthcare.

    Yao Shang is a CA in Michael’s external mandates course where she has researched performance attribution, including the modeling of complex path-dependent cost and fee structures and their effect on manager incentivization and performance, and highly realistic backtesting, including the implementation of agent-based borrowing and lending to simulate the effects of cash drag and forced liquidations. She has worked with wealth managers at Morgan Stanley using deep learning techniques and holds several degrees from Rensselaer Polytechnic Institute. She will receive a Master’s in Operations Research from Columbia University in May of 2021.

    Recorded: 16 Mar 2021

    Hello, everyone and welcome to this webinar. My name is Alex Roumi. And I work in the marketing group at MathWorks. In today's webinar, I will be walking you through the strategy backtesting framework of Financial Toolbox. I'm extremely excited to welcome Michael, Malin, and Yao from Columbia University. And they will tell us how the backtesting framework was used for global tactical asset allocation.

    In today's agenda, we will explain briefly what backtesting is. Go over the backtesting framework of financial toolbox, and show how it can be implemented in MATLAB. After that, Michael and his team will show us how the backtesting framework was used for global tactical asset allocation and Columbia University environmental, social, and governance machine learning research at the large European hedge fund.

    When you think of the life of an investment strategy, first, there is the basket of stocks that you plan to invest in. The quants then do some idea generation, where they do some prototyping, some research, and pick out one or many investment strategies to test them out. Now, the question the portfolio manager is going to have, which of these strategies are viable? Which should we pick?

    In order to test that out, the strategies are implemented in code. And then finally, test them out. That's where backtesting comes into play. You test and compare the strategies without risking real money.

    Let's assume we came up with a portfolio of three assets. And we already know how each of these assets performed in the past. Our goal is to test a trading strategy, which is the logic used to decide how to allocate assets in our portfolio. Should I invest equally in each asset or not?

    The strategy is tested over historical data. It can be tested during an economic recession, like the 2008 one, or an economic expansion. So we would like to see how our portfolio would have performed if we had applied these strategies in a variety of market conditions. There are numerous challenges in backtesting, such as getting data into the right format. Some of your data might come in daily, like your closing prices, and others you might get quarterly, like your macroeconomic numbers. Or sometimes data are incomplete, such as ESG data.

    Other challenges include over-fitting and look-ahead bias. Building a backtesting infrastructure from scratch can be very time consuming. And on the other hand, back-box backtesting tools that might not let you customize your test to the level that you would like, for example, to specify a custom transaction cost model. And now, you're not restricted to your traditional financial signals, you want to include things like sentiment analysis and specify machine learning based investment strategies as well.

    In this demo, we will see how to perform backtesting of five portfolio strategies and compare them, such as equal weighted, where the invested capital is shared equally among the assets. In the maximization of your Sharpe ratio strategy, we calculate the portfolio weights that maximize the Sharpe ratio, which is also the tangency portfolio on the efficient frontier.

    In the inverse variance strategy, the weights are proportional to the inverse of the asset variance. In this way, assets which have smaller volatility are given more weight. The Markowitz portfolio optimization aims to maximize the return and minimize the risk for a fixed risk aversion coefficient. For example, given two portfolios that offer the same expected return, the investor will prefer the less risky one.

    Finally, the hierarchical risk parity or HRP consists of computing a hierarchical tree from the correlation matrix. And then diversifying the weights across the different clusters. You can check the documentation to learn more and go in depth in each of these strategies.

    These strategies are backtested using a backtesting framework implemented in MATLAB, which consists of backtest strategy and backtest engine objects and run backtest and summary functions. You can compare the strategies and check high-level results of each strategy from the backtest, such as the total return, the Sharpe ratio, the max drawdown, et cetera. You can plot the equity curve of each strategy to visualize their performance over the backtest or any other output, such as comparing the transaction costs of each method.

    Before jumping into MATLAB, let's highlight the backtesting workflow. We define initially the investment strategies. Each strategy is implemented in a rebalance function, such as equal weighted inverse variance, hierarchical risk parity HRP, et cetera. In other terms, each strategy decides how to divide your money among a variety of assets. You can also specify the size of the rolling window, or how often the portfolio is rebalanced, or specify a custom model for the transaction costs using the parameters shown here.

    The backtest engine takes as input the strategies defined earlier, as well as other parameters, such as the risk-free rate, the cash borrow rate, or the initial portfolio value. The backtest is run over the timetable of adjusted asset price data and, optionally, any trading signal. Such information can include technical indicators, sentiment analysis, and so on.

    Finally, the summary function generates a table of strategy performance results for the backtest. From there, you can plot the metrics of your choice, such as comparing the average turnover for each strategy. The workflow of our demo is the following. Start by loading the adjusted price data of the stocks. Build the investment strategies, and use the backtesting framework to backtest the strategies. And finally, summarize the results and compare the strategy's performance.

    Here's a script I have already written in Live Editor to backtest the investment strategies. In this example, we will develop five investment strategies and then compare their performance after running over a one year of historical stock data. First, load one year of adjusted price data for 30 stocks from the component stocks of the Dow Jones Industrial average.

    The backtesting framework requires adjusted asset prices. Meaning prices are adjusted for dividends and splits. The prices are stored in a MATLAB timetable where each column is a time series for an investable asset.

    Let's prune the table and display only these 15 stocks for addability. Prices has one year of adjusted price data for 15 stocks. The code logic of each strategy is implemented in a rebalanced function, which specifies how a strategy allocates capital in a portfolio.

    We can see here the equal weighting, the maximization of Sharpe ratio strategy, the inverse variance, et cetera. I'm not going into details with each one of these as we have examples and functionalities in the doc for all these strategies and more. But as an overview, a rebalanced function uses the current weights and the rolling windows of asset prices to compute the desired new portfolio weights. Also note that local functions in Live Editor are placed at the end of the code.

    We will use the strategy rebalancing functions to compute the initial weights for each strategy. This example uses the first two months of the data set to initialize the strategies. The backtest is then run over the remaining data.

    At the beginning of the investment, we have 100% in cash and 0% in stocks. So the current weights are zero. To compute the initial weights, we call the rebalance function that takes as input the current weight, which is zero as we still don't have any stocks in hand, and the first two months of price data that the strategies will use to set the desired weights.

    Setting the initial weights is important because, otherwise, the strategies begin to backtest with 100% in cash, earning the risk-free rate until the first rebalance date. We can see here, the initial weight for each strategy. Obviously, stocks are equally weighted in the equal weighted strategy.

    First step of the workflow is to define the backtest strategy class. The backtest strategy class takes as input the name and rebalancing function for each strategy. I defined here the five strategies I mentioned earlier. Additionally, you can specify other parameters, such as a rebalance frequency to determine how often the backtesting engine rebalances and reallocates the portfolio. In our example, we are rebalancing our portfolio every month. The rebalance frequency is a fixed predetermined number and is independent from the market direction.

    The look back window is the chunk of data from the price timetable passed to the rebalance function to determine the new weights. So in our example, we are rebalancing every month. And the rebalance functions will use a rolling window of asset prices that will contain at least 40 days but no more than 126 days of price data, depending on the amount of available price history at each rebalanced date.

    In this example, the transaction costs are fixed at 25 basis points. But you can build a variable transaction cost model and pass it to the backtest strategy class. Next, we create the backtest engine object to backtest the strategies.

    The backtest engine takes as input the strategies defined earlier. Additionally, you can set other parameters, such as the risk-free rate. The default behavior for the risk-free rate is to follow the rate convention. If the rate convention is set to annualize, then the rate is interpreted as annualized. Therefore, you don't need to divide your annualized rates into per time step rates based on the frequency of the pricing data. The initial portfolio value here is $10,000. It's a portfolio that starts growing from this initial value.

    Third, we run the backtest on the strategies. The run backtest function executes the backtest using the test data partition. You can optionally specify signal data when running a backtest, such as technical indicators, sentiment analysis, or ESG factors. These indicators make buy and sell decisions. You can check this example in the doc to learn more how to backtest with trading signals. We run the backtest skipping the warm-up period to avoid look-ahead bias.

    Last step is to summarize the results. Finally, use the summary function to display the results of the backtest and generate a table containing the performance of the strategies. The table contains high-level results from the backtest, where each row of the table is an output measurement of the performance strategy, such as total return, Sharpe ratio, max turnover, et cetera. And each column occupies a strategy.

    If you want to examine detailed results of the backtest, check the backtest engine object. You can examine daily values for portfolio returns, asset positions, turnover, and transaction costs. For example, you can display the simple return at each time step and generate a histogram of the daily portfolio returns.

    Use equity curve to plot the equity curve for the five different investment strategies and see how the five strategies perform. It's simply a visual representation of how your portfolio has grown over time. The positive slope indicates that the trading strategies are profitable, while a negative slope shows that they are generating a negative return. You can also make lots of certain metrics, such as comparing the strategies turnover or transaction costs. You can visualize the change in the strategy allocations over time using an area chart of the daily asset positions.

    One of the biggest pains to do the backtesting is getting data into the right format. Placing data into a timetable helps align and synchronize data with different time intervals. Also, you can clean your data using Live Editor. In backtesting, you don't have to use historical data, but you can use simulated market data and do the backtesting over stress scenarios and check how the strategies perform.

    With this framework, you can plug and play different parts into it by changing the rebalance function, or build and test any strategy of your choice, or simply integrate a transaction cost model into the framework, automate the workflow, and visualize the results. You can also integrate trading signals to solve efficient portfolios with ESG or other signals, such as machine learning models, sentiment analysis, or any technical indicators. I'll hand it over now to Michael. Michael?

    Hello, I'm Michael Robbins. I teach several courses in quantitative asset management to graduate students at Columbia University and writing a book, Quantitative Asset Management for McGraw-Hill to be published in 2022. The website for the book is quantitativeassetmanagement.com, where there will be quite a bit of MATLAB code, which we'll discuss today, and is also discussed in depth in the book. And you can ask questions about today's presentation at info@quantitativeassetmanagement.com.

    I brought two students with me today to help describe how we use MATLAB's backtesting framework in the courses. I've been the chief investment officer of five large investment firms and the chief investment risk officer of a large pension system. And I've taught for Columbia University for quite a few years now.

    Malin has taken my environmental social and governance course and has been a CA in my global tactical allocations course. Yao took my external managers course and is now a CA in that course. We won't be able to discuss the courses in detail. We'll provide a high-level overview of some of the slides. If you want to learn more about the courses, you can read my book. I go into them in quite a bit of detail.

    Today, we'll talk about an overview of how we modified MATLAB's code and the ESG and GTAA courses, as well as the external managers course. The programming for the course has all followed the same general framework, where we gather a lot of proprietary data. We organize it in the database. We perform machine learning on that data as inputs into the backtesting framework. And then after running the backtester, we use performance attribution to study the results.

    I'll talk about how we modified the code. Malin will talk about ESG and GTAA. And Yao will talk about external managers.

    From the very beginning, it's been very important to us to use vectorized inputs in the backtesting framework and not just the event-driven code. This is for a few reasons. Very importantly, most of the students use the language of their choice to do the factor engineering and provide that as an input, rather than programming it into the rebalance function. It also allowed us to use technologies that were cumbersome to use inside the backtesting framework, or just took too long to calculate in that framework.

    Technologies, like MATLAB's Classification Learner app, Amazon's Prophet, and Facebook's PyTouch. It also allowed us to modify the code to use optional input data, like volume and open interest data, to be used for things like calculating market impact. We modified the code to pass that optional data directly into the cost and rebalance functions to use in sophisticated calculations.

    The backtesting engine runs in a loop and calls the rebalance and cost functions at each rebalance iteration. The rebalance function creates buy and sell requests to be evaluated by the cost function. Looking at the price of the day, the rebalance function might, for instance, say buy 100 shares of ABC. The rebalance function is past that data, along with the position sizes. A modification we made to the backtesting code.

    The cost function then evaluates the market impact and transaction fees. It also decides how much of the transaction can be executed. For instance, it might decide that the price of the purchase should be $25 instead of $20. And that the full 100 share lot is not reasonable to execute given, for instance, the volume of the day. It passes that information back to the backtesting engine, a modification we made to the code.

    It might say, for instance, you cannot transact 100 shares. You have to only transact 90. We've modified the backtesting engine to transact only amount that the cost function allows and not the amount that the rebalance function requested. We also modified it to perform other calculations that were very important, including fee structures that were complicated by path dependency.

    We included management fees, incentive fees, hurdle rates, and high watermarks, as well as other complications that we needed in order to do our analysis. In an actual fund, it's important to include all these things in your simulations because they need to be provided to the clients to show the performance of the fund. The fund's performance is not useful without these intricate calculations.

    We also included an overdraft account that used time varying lending rates. This was very important for two reasons. If we wanted to make purchases that exceeded the amount of money in our cash account, we wanted to penalize the rebalance function for not setting aside enough cash by charging the lending rate. We also did not want to adjust the percentages of the other investments automatically. We wanted the rebalance function to explicitly request sales of other assets.

    So instead of automatically reducing the percentages of other assets, we borrowed money from the overdraft account. This also allowed us to use leverage in our portfolios, which is very important for a number of portfolio managers. It allowed us to purchase more assets than we had cash for by borrowing money at the lending rate. This lending rate continued to be used until the rebalance function explicitly paid back the overdraft account, allowing us to have persistent leverage.

    Another important adjustment that we made to the backtesting framework is to provide independent profit and loss streams for each asset. This provided a number of additional abilities that the backtesting engine didn't have, including being able to analyze those performance streams in the rebalance function to do things like stop losses. And also, very importantly, to do asset-level analysis like Brinson performance attribution after the backtesting analysis was done.

    Using all these adjustments that we made to the rebalance engine, we were able to integrate with the MATLAB statistics and machine learning toolbox and the deep learning toolbox to make the inputs. The data feed toolbox, in order to actually transact with companies like interactive brokers. The transaction cost analysis tool box in the cost function in order to do sophisticated market impact analysis. And to use a multi-period Brinson performance attribution at the end. We ran all these what-if analyses in a large loop and produced thousands of outputs to be analyzed later.

    Now, Malin will discuss how we used these analyses in the ESG and GTAA classes.

    Thank you, Michael. So over the past year or so, I have worked on a wide variety of projects together with Michael. And the very first one was an ESG analyses.

    This was for a fund with 193 employees in seven offices. So this is a sophisticated fund who specialized in ESG factors. And they provided us with the proprietary data, as well as their returns. And throughout the project, the team and I used machine learning and backtesting to evaluate the factors and identify any potential areas for improvement.

    So the first thing we had to do was to make sure that the real performance was reconciled with the simulations. So in order to analyze our data, we needed to compare the actual performance of investment portfolios to the simulated performance of those portfolios under different conditions. So for example, with or without tax harvesting. So we took the original allocations, and we used that as an input and ran through the backtester to make sure that we got the same results as in reality.

    And of course, in order to do this, we needed software. So this is where MathWorks came in. And they built a really excellent tool for us. However, it lacked a few features that we needed.

    So the first requirement was that we could use the vectorized input to prove out the original performance before simulating the what-ifs. So after completing this, we moved on. And we used the triple-barrier method to build our response set and walk-forward analysis to evaluate our strategies. We also used the Classification Learner app to train models on the data and to hyper-parameterize the models.

    For the walk-forward analyses, we use quarantine gaps between the training and the test set. And we did this in order to minimize the possibility of auto-correlation between these two periods. And both the backtester and the look-forward helped us realize a pattern of decreasing efficacy. So this was one of many, many ways that the backtester proved incredibly helpful for the team, as we were able to identify this behavior.

    We also explored risk control overlays throughout the project. And after wrapping up the ESG project, we moved on to look at global macroeconomic factors. And for this, we worked in different teams. And we were competing for the model predictive models.

    So looking at the tree chart here, each node represents a team. And that team had a decision for which out of two assets to buy or sell. So looking at the top left corner here, for example, commodities versus equities, the team there had to run models to decide on whether to buy or sell commodities or equities. At the end of the project, we combined all of these results, and we built a global tactical asset allocation strategy. And for this, we used the same methods that we had for the ESG project.

    And last, but definitely not least, Michael built an app on the Bloomberg terminals on campus from which students can directly download data from Bloomberg and run different machine learning algorithms. And with that, I will pass it over to Yao.

    Next, I will talk about the analysis we did on the data of 101 external managers. A data company generously offered a rich and comprehensive database of global advisor holdings and performance to provide an in-depth look inside the portfolios. They produced this by funding accounts with robo-advisors to track the performance on those accounts.

    And they asked us several questions. We answered those questions using the data they provided. The first question they have is, how do the fund compare to each other? For instance, compare all the taxable funds to each other, all the social investment funds to each other.

    The second question they have is, how all the funds' performance change if the rebalance frequency was altered. For instance, if a dividend investing fund were to wait until their rebalance date to invest or if a tax-loss harvesting fund were to wait until their rebalance date? And the third question is, how would funds' performance change if their composition were altered? For example, what happens if large cash allocations were reduced to be similar to that of their peers on a risk adjusted basis?

    To answer these questions, we need to reconcile the real performance with the simulated results. We use the vectorized input, and that is the signal table, to provide the original performance before simulating our what-if questions. To accomplish this, we used the actual fund asset allocation as the signal file and the rebalance function. And rebalanced the portfolio to the signal weights. We also performed benchmark analysis. We reverse engineered a policy portfolio, a benchmark, and used those allocations in a signal file to simulate the benchmark returns and attributions.

    One of the enhancements we added is to split the return stream for attribution in stop-loss analysis. The original backtester provided returns and costs for the portfolio as a whole, but we needed to divide these return streams and record them for each asset. This was required for the final multi-period Brinson attribution analysis and also for some rebalance logic, like stop-loss orders.

    As easy as it may sound, it is not a trivial task. Besides splitting the return stream, we also added a second cash account. And we call that the overdraft account. The original backtester didn't distinguish between lending and borrowing. And that's why we need an overdraft account.

    This allowed us to transact when the cash was used up without selling other stocks to generate cash. Also, this allowed us to stimulate levered funds. Like, if we want to borrow 20% of the assets in the fund, we'll have a 120 to 20 structure. Another enhancement is we added complex cost and fee structures, including incentive fees, management fees, crystallization dates, hurdle rates, and high watermarks.

    This complex cost structure enabled us to do more interesting analysis. For example, we can do past dependant fees and costs for simulations. Also, we can perform multi-period Brinson for performance and risk attribution analysis. Here's a comparison between the summary function of the original backtester and the enhanced backtester. The one on the left is limited to the portfolio level results. And the enhanced backtester can disaggregated the results using the multi-period Brinson method.

    With all the great features that we added, here are still a lot on our to-do list. And here are several questions that we want to answer. The first one is, how did each portfolio perform outright and compared to its benchmark using a variety of portfolio-level measures and fund-level Brinson analysis? The second question is, how did portfolios compare to each other in groups? For instance, all social fund outright versus each other and relative to their benchmarks.

    The third one is, how did groups compare? For instance, the average social funds versus the average taxable funds, a particular manager's social fund versus his taxable fund outright, the social performance versus their benchmark. The fourth question we want to answer is, how did portfolios compare a risk-adjusted basis? For instance, use CAPM to leverage the portfolios to an equal level. Adjust the equity to a fixed income ratio to equalize risk and simulate the results.

    And the last question is, was tax-loss harvesting actually effective? For example, compare a manager's tax-loss harvesting funds to their taxable fund. And again, after eliminating the tax-loss harvesting rebalancing date. Also do this after estimating taxes as well.

    And that concludes our presentation. Thank you so much for listening. If you have any questions, please send us emails. And if you want more information, please visit the website. All of the contact information is on the upper right corner.

    View more related videos