Can Yield Curves Predict Stock Market Direction? - A Machine Learning Perspective

August 19th, 2019

Save time and make investing easy

Investing can be so rewarding, but also time consuming and stressful. Passiv is here to save you time and make investing easy.

Get Started

This blog post was originally published on the blog by Jin Choi. The website no longer exists, but Jin has graciously allowed us to re-publish his research for the benefit of future investors forever.

Image Credit:dani3315/

Many investors are concerned about the yield curve today. Generally speaking, a positive sloping yield curve (i.e. where long term interest rates are higher than short term rates) portends future economic health. By contrast, negative sloping yield curves (a.k.a. inverted curves) have tended to precede recessions.

Today, the yield curve is the most inverted its been since before the financial crisis. This has led some investors to wonder: should they sell stocks today to protect their portfolios? Or should they hold onto them because, despite widespread belief, yield curves may not be the predictor they are often made out to be?

Academic Paper And Its Problems

At least one set of academics has come out with a purported answer. Just last month, Fama and French published a draft paper that examined a market timing strategy based on yield curves. The strategies they examined gradually unloaded stocks when yield curves were inverted, and gradually added them back when yield curves sloped upwards. They found that such strategies ended up underperforming the simple buy and hold strategy where the investor never sold stocks.

Personally, I found the paper to be wanting. I know thats a bold thing to say considering one of the authors - Eugene Fama - is a Nobel laureate. But, hear me out. All that the paper proved was that the specific strategies devised by Fama and French didnt work. This might be okay if the strategies made good use of the yield curves, but Im not convinced.

The chart below shows the ten to two year treasury yield spreads. As you can see, the spread tended to go negative (i.e. yield curve inverted) before a recession. However, there tended to be significant lags between the timing of the inversion and the beginning of the recession. For example, the yield curve inverted the last time in 2006, a full two years before the recession started.

Stock prices reacted more or less in sync with economic health during the last recession, rising until 2007, and falling big in 2008. But if you had followed Fama and Frenchs strategy, you would have sold stocks until 2007, and started buying them back in 2008, which is the wrong strategy to follow. Does this mean that we shouldnt use yield curves to inform our investing decisions? No, it just means we shouldnt use it the specific way Fama and French outlined.

If you want to know if yield curves can be used in any fashion to make investment decisions, its better to see if yield curves have any predictive power over the stock market. To answer whether thats the case or not, I turned to machine learning.

My Machine Learning Model

Now, before we go any further, let me warn you - what Im about to show you doesnt qualify as rigorous academic research. If I wanted to make this a proper academic study, I would conduct a lot more tests, use more data, etc. This is more like tinkering in a lab.

In order to see if theres any connection between yield curves and stock prices, I first downloaded two data sets from Quandl - historical treasury yields of different maturities, and the monthly performance of the S&P 500.

Since I was interested in whether the shape of the yield curve predicted stock prices, I constructed a time series consisting of the following interest rate differentials.

  • 1yr - 3mo
  • 2yr - 1yr
  • 5yr - 2yr
  • 10yr - 3mo
  • 10yr - 2yr
  • 30yr - 3mo
  • 30yr - 2yr
  • 30yr - 10yr

I didnt choose these differentials based on any rigorous scientific reasons. Rather, I guessed that these would contain the essence of the shape of the yield curve. These data points constituted the inputs to the machine learning model.

The target output of the model consisted of the ensuing 12 month S&P 500 returns in excess of the 3 month treasury bill interest rates. This is consistent with what Fama and French used in their paper.

I then divided the data set into two - one for training, and the other for testing. The training set consisted of all data until the end of 2009. The test set consisted of all data thereafter. There were 229 data points in the training set, and 104 data points in the test set. Experienced data scientists will note that this is a rather simplistic way to evaluate the model. If I were to do more rigorous research, Id use a method like the combinatorial purged cross validation. But as I said, Im just tinkering here.

With the inputs and targets defined, I pondered which machine learning model to use, and I decided that some type of RNN (Recurrent Neural Network) made the most sense. I had a theory that in addition to the current shape of the yield curve, the evolution of the yield curve mattered as well, and RNNs had the ability to make use of the evolutionary information. In order to make use of RNNs, I modified the inputs such that each input consisted of 12 months worth of yield curve history.

Unfortunately, I discovered that many of the popular types of RNN didnt work for this problem. With only 229 data points available for training, the more sophisticated RNNs like GRU and LSTM persistently overfit, even when using only 1 output unit. The only RNN that didnt overfit to uselessness was the simple RNN, with 1 output unit.

Fortunately, the simple RNN seems to work decently well. The graph below shows a set of predicted S&P 500 returns vs. actual for the test data set.

As you can see, though the predictions are not super accurate, it suggests that the model does appear to have some predictive power. The slope of the line is 0.48, which suggests an R squared of 23% - a fairly high number. However, I suspect that this overstates the models predictive capability. Indeed, if you tweak the model just a little bit, then you end up with a somewhat lower R squared. For example, if I lower the batch size of training from 8 to 4, I get an R squared of roughly 10% instead.

But even an R squared of 10% is significant, and it suggests that yield curves can be used in some fashion to get an idea of where the stock market is headed next. I also suspect that yield curve data would be even more useful if used in conjunction with other data, such as inflation data. If this is true, I believe investors would be able to craft some investment strategy that uses yield curves to beat the buy and hold strategy outlined in Fama and Frenchs paper.

But before we get too excited, let me reiterate that theres no guarantee my model does work. It appears to work at first glance, but really confirming that would take more effort, and Im unwilling to put in that effort. If someone would like to continue where I left off, feel free to do so. Ive made my code available for download here in the Jupyter notebook format. If you do use my code, I would appreciate it if you could credit me.

Finally, youre probably curious about what the model says about where stock prices are going. I do have an answer for you. It says stocks will go down slightly (roughly -3%) in the next 12 months. I think this is in line with many investors gut feelings on where stock prices are headed.

This blog post was originally published on the blog by Jin Choi. The website no longer exists, but Jin has graciously allowed us to re-publish his research for the benefit of future investors forever.

Get all the insider financial info